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OBJECTIVE 


The  components  integral  to  the  operation  of  an  Electronic  Crewmember  have  started  to  take 
shape.  Questions  have  been  raised  as  to  the  nature  of  the  EC  when  finished.  What  are  the  key 
components  that  will  ensure  a  successful  emergence  of  this  new  technology?  How  can  we  plan 
for  their  development  and  incorporate  the  software  and  hardware  elements  to  function  in  concert 
with  one  another.  The  purpose  of  this  workshop  was  to  examine  these  concerns. 

This  meeting  was  a  follow  up  to  previously  successful  meetings  (1988,  1990  and  1994)  co¬ 
sponsored  by  the  German  Air  Force,  Royal  Air  Force,  and  the  US  Air  Force  Wright  Laboratory 
and  European  Office  of  Aerospace  Research  and  Development.  This  fourth  meeting  provided  a 
valuable  forum  for  the  experts  of  several  countries  to  measure  progress  in  this  critical  technical 
area.  It  also  allowed  for  the  exchange  of  new  ideas,  concepts,  and  data  relative  to  hardware  and 
software  capabilities  that  can  be  included  in  an  aircraft  system  design,  to  aid  the  human  operator 
to  perform  the  mission.  Attendance  at  this  workshop  was  by  invitation  only.  It  brought  together 
experts  representing  cockpit  design  disciplines  including  hardware  and  software  technologists,  as 
well  as  human  factors  specialists,  and  pilots  to  wrestle  with  questions  such  as: 

(1)  What  are  the  core  qualities  that  the  Electronic  Crewmember  must  possess? 

(2)  How  does  one  estimate  the  amount  of  software  code  involved? 

(3)  What  are  the  key  software  modules? 

(4)  What  is  necessary  to  ensure  the  modules  function  symbiotically? 

(5)  What  is  sufficient  functionality  within  the  Electronic  Crewmember  to  satisfy  the  human 

operator  requirements? 

The  workshop  comprised  formal  paper  sessions  and  small  group  discussions.  Proceedings  will  be 
published  in  reports  by  the  sponsoring  laboratories.  The  numbers  of  persons  attending  was 
restricted  to  60.  All  invited  attendees  were  expected  to  contribute  through  formal  presentations 
or  through  active  participation  in  the  meeting  discussions.  The  selection  of  invitees  was  based 
on  the  quality  of  the  proposed  contributions.  An  attempt  was  made  to  balance  representation 
across  nations  as  well  as  across  themes. 


IX 
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WORKSHOP  BACKGROUND 


Ever  since  the  movie  Star  Wars  showed  Luke  Skywalker  and  R2D2  teaming  up  to  destroy  the 
Death  Star,  there  has  been  considerable  speculation  as  to  how  an  efficient  pilot-robot  team  could 
be  created.  Since  weight  is  a  critical  design  factor  in  airborne  systems,  the  literal  building  of  a 
pilot-robot  team  has  not  been  undertaken;  rather,  the  emphasis  shifted  to  incorporating  the 
intelligence  of  the  robot.  As  work  in  this  area  progressed,  such  terms  as  “electronic 
crewmember”  and  “black  box  back  seater”  began  to  enter  the  vocabulary  of  both  the  crewstation 
design  and  computer  software  communities.  While  the  use  of  these  titles  served  to  stimulate 
thinking  in  the  area  of  human-computer  teamwork,  a  major  program  was  required  to  start  the 
design  and  implementation  of  concepts  needed  to  build  an  electronie  crewmember  (EC);  in  the 
US  this  took  the  form  of  the  Pilot’s  Associate(PA)  Program.  The  establishment  of  the  PA 
Program  in  1985  gave  credence  to  the  idea  that  the  building  of  the  brain  of  R2D2,  in  some  very 
simplified  form,  might  be  possible.  The  some  of  the  results  of  this  program  have  been 
transitioned  to  the  US  Army’s  Rotrary  Pilot’s  Associate  Program  which  continues  to  strive  for 
the  same  goal.  In  Europe,  AI  efforts  have  centered  around  a  number  of  programs.  These  include 
the  French  "Co-pilote  Electronique  ",  the  British  Mission  Management  Aid  (MMA),  and  the 
German  Crew  Assistant  Military  Aircraft  (CAMA)  Cockpit  Assistant  Systems.  They  too  have 
tried  to  achieve  the  goal  of  human  computer  teamwork  in  the  cockpit. 

In  the  next  two  years,  numerous  discussions  were  held  to  explore  some  of  cockpit  ramifications 
created  by  the  use  of  a  pilot-EC  team  within  the  aircraft.  These  diseussions  occurred  in  various 
technical  meetings  within  the  US  and  the  UK.  In  one  of  the  meetings  held  in  the  US,  attended  by 
representatives  of  the  Air  Force  of  the  then  Federal  Republic  of  Germany  (FRG),  as  well  as  US 
and  UK  representatives,  the  idea  of  the  initial  workshop  was  bom.  Although  progress  on  the 
idea  of  a  workshop  concerning  human-EC  teamwork  eontinued,  in  1987  an  event  occurred  which 
demonstrated  the  definite  need  for  the  workshop. 

In  April  of  1987,  USAF  representatives  gave  a  paper  at  a  meeting  of  the  Royal  Aeronautical 
Society  in  London  and  again  at  a  meeting  of  the  Ergonomics  Society  in  Swansea,  Wales.  The 
subject  of  the  paper  was  “Workload  and  Situation  Awareness  in  Future  Aircraft”,  and  a  section 
of  the  paper  discussed  workload  sharing  between  the  pilot  and  the  EC.  During  both  meetings  the 
same  kinds  of  questions  were  asked:  Is  the  pilot  always  in  charge?  Can  the  pilot  and  EC  really 
be  called  a  team?  Why  do  you  need  the  pilot  at  all? 

These  thought  provoking  questions  resulted  in  continued  discussions  with  technical  personnel  in 
the  US,  UK  and  FRG,  and  the  result  was  the  1988  workshop  entitled,  “The  Human-Electronic 
Crew:  Can  They  Work  Together”.  The  workshop  was  documented  by  the  Royal  Air  Force  (RAF 
lAM  BSD-DR-G4,  Dec  1988,  and  the  U.S.  Air  Force  WRDC-TR-89-7008,  Jul  1989).  Following 
the  1988  workshop,  interest  was  expressed  in  holding  an  additional  meeting  on  the  topic  of 
human-electronic  teamwork.  The  result  was  a  1990  workshop  entitled,  “The  Human-Electronic 
Crew;  Is  the  Team  Maturing”,  (RAF  lAM  PD-DR-P5,  April  1991;WL-TR-92-3078,  July  1992). 
The  1988  and  1990  workshops  were  sponsored  by  the  USAF  European  Office  of  Aerospace 
Research  and  Development  (EOARD),  and  hosted  very  generously  by  the  German  Air  Force. 
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There  was  a  four  year  hiatus  between  the  second  and  third  workshops.  Events  relating  to  end  of 
the  Cold  War  caused  a  very  djTiamic  environment,  with  many  governmental  reorganizations 
occurring  on  both  sides  of  the  Atlantic.  After  these  events  were  sorted  out,  plans  began  to 
convene  the  third  workshop.  Sponsorship  was  obtained  from  BOARD,  and  resulted  in  the  third 
workshop,  which  the  Royal  Air  Force  hosted  in  Cambridge,  England.,  "The  Human-Electronic 
‘  Crew:  Can  We  Trust  the  Team?  (DRA/CHS/HS3HR95001/01,  Jan  1995;  WL-TR-96-3039,  Dec 
1995). 

The  fourth  of  the  workshop  series,  which  is  described  in  these  proceedings,  was  again  sponsored 
by  BOARD  and  was  held  in  Kreuth,  Germany  in  1997.  The  theme  of  this  conference  was  "The 
Human  Electronic-Crew:  the  Right  Stuff?",  and  the  German  Air  Force,  Universitat  der 
Bundeswehr,  graciously  served  as  host. 


EXECUTIVE  SUMMARY 


The  meeting  was  divided  into  two  sections:  formal  presentations  (papers)  and  workshop.  The  29 
papers,  presented  by  representatives  from  seven  different  countries,  fit  into  four  major 
categories:  mission  descriptions,  design  methodology,  human-electronic  crew  interface 
technologies,  and  the  use  of  an  EC  in  the  commercial  aircraft  arena.  A  summary  of  the  ideas 
from  the  papers  is  given  below. 

Papers 

The  papers  covered  how  to  communicate  with  the  eleetronic  crewmember  through  automatic 
speech  recognition,  and  also  discussed  display  formats  in  order  to  show  the  output  of  the  EC. 
Applications  were  shown  in  both  an  airborne  and  naval  environments.  Detailed  applications, 
which  presented  two  decision  aids  in  great  detail,  were  demonstrated  in  a  French  discussion  of 
The  Copilote  Electronique  and  a  German  presentation  on  The  Crew  Assistant  Military  Aircraft 
(CAMA). 

The  topic  of  the  application  of  the  EC  to  commercial  aircraft  was  also  discussed  in  detail.  The 
demonstration  of  one  small  version  of  the  EC  which  could  be  applied  in  this  environment,  The 
Hazard  Monitor,  showed  that  the  EC  does  have  potential  in  this  area.  However,  the  issue  of 
certification  of  the  EC  in  the  eommercial  aircraft  environment  remains  a  very  important  concern. 

A  step  forward  in  the  goal  of  building  the  Human-Electronic  Crewmember  team  was  presented 
in  a  series  of  papers.  One  dealt  with  a  new  way  to  communicate  with  the  EC  using  shortcut 
language  called  plays  (used  extensively  in  American  football).  Others  discussed  the  functional 
architecture  for  constructing  the  EC,  as  well  as  the  “cognitive  architectures"  of  the  pilot.  Both  of 
these  features  could  be  crucial  in  the  effort  to  build  a  teaming  relationship  between  the  EC  and 
the  operator. 

Workshop 

After  the  presentation  of  the  papers,  the  second  half  of  the  meeting  consisted  of  a  workshop;  its 
purpose  was  to  form  five  teams  to  deal  with  AI  technology  and  cockpit  implications  of  the 
technology.  The  teams  were  composed  of  three  technical  disciplines  represented  at  the 
conference  ~  aircrew,  crewstation  designers,  and  artificial  intelligence  experts.  At  the  end  of  the 
workshop,  each  of  the  five  team  leaders  presented  the  results  of  their  deliberations.  The  details 
are  documented  in  the  Group  Discussions  portion  of  these  proceedings;  a  summary  is  presented 
below. 

There  were  two  key  conclusion  which  resulted  from  the  workshop  committees.  The  first  was 
that  there  is  definite  interest  in  an  EC  for  commercial  aircraft.  However,  the  certification  of  an 
EC  in  the  commercial  arena  may  be  much  more  difficult  than  in  the  military  arena  because  of 
safety  considerations.  The  second  conclusion  was  that  as  conventional  avionics  become  more 
sophisticated,  and  more  "mini-EC"  decision  aids  come  online,  many  parts  of  the  EC  may  be 
created  from  the  bottom  up.  The  key  component  that  may  remain  in  the  creation  of  the  EC  is 
the  software  supervising  the  activities  of  the  subparts. 
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CONCLUSION 


Besides  the  technical  information  gathered,  one  of  the  major  accomplishments  was  the  positive 
interchange  among  the  participants.  There  was  a  genuine  sharing  of  information  and  ideas  in 
order  to  attack  the  common  problem  of  information  overload  in  the  cockpit.  The  participating 
countries  are  striving  to  reach  a  common  goal,  and  the  ideas  exchanged  in  the  Workshop  should 
prove  very  beneficial  to  all  of  them. 


KEYNOTE  ADDRESS 

UK  MOD  OPERATIONAL  REQUIREMENTS 

by 

Squadron  Leader  M  W  G  Hopkins  MBE  MA  RAF 
Operational  Requirements  (Air) 

Ministry  of  Defence 


I  should  like  to  thank  Reiner  Onken,  Bob 
Taylor  and  John  Reising,  the  Workshop  Co- 
Directors,  for  inviting  me  to  give  the  UK 
MOD  perspective  and  Operational 
Requirements  for  the  Human-Electronic 
Crew.  I  am  grateful  to  the  workshop  for 
allowing  me  to  follow  in  Gp  Capt  Dusty 
Miller’s  footsteps  and,  hopefully,  continue 
with  the  line  he  put  forward  in  his  opening 
address  in  Cambridge  in  1994.  As  part  of 
the  MOD  sponsor  team  for  Air  Human 
Factors  research,  I  am  particularly  pleased 
that  such  a  workshop  should  be  taking  plaee. 
In  these  days  of  financial  restraint  in  all 
matters  military,  and  especially  research,  it 
is  crucial  that  we  should  be  maximising  the 
talents  and  resources  of  our  respective 
nations  -  we  can  no  longer  work  in  isolation 
if  we  are  to  influence  cockpits  of  the  future 
and,  indeed,  get  them  right.  As  a  front-line 
pilot  (and  customer)  I  do  not  propose  to 
lecture  the  considerable  intellects  present  on 
the  technological  and  psychological  issues 
but  to  try  and  bring  to  bear  some  of  my 
operational  experience.  It  is  this  kind  of 
input  that  I  believe  to  be  crucial,  not  only  in 
this  area,  but  throughout  military-focused 
research  if  we  (the  military)  are  to  get  value 
for  our  increasingly  limited  budget  whilst 
retaining  the  military  effectiveness  -  in 
short,  the  edge. 

There  is  no  doubt  that,  as  the  information 
revolution  gathers  pace,  the  cockpit,  and  the 
command  and  control  aspects,  will  dominate 
the  design  and  integration  of  future  aircraft. 
Gone  are  the  days  of  engineers  designing 
cockpits  on  their  own  -  it  is  now  the  domain 
of  a  team  of  engineers,  human  factors 
specialists  and  aircrew  striving  to  achieve 
the  optimum  balance.  I  am  confident  this 
teaming  will  and  must  continue  if  we  are  to 


satisfy  the  only  non-variable  in  the  equation 
-  the  human  -  although  I  accept  that  there  is 
much  variation  and  unpredictability  within 
the  species. 

Questions  I  am  often  asked  are:  will  future 
combat  aircraft  have  a  cockpit  and  what 
form  will  the  future  inhabited  cockpit  take? 

To  answer  the  first  of  those  questions  - 1 
believe  the  manned  combat  aircraft  is  here 
to  stay  for  some  considerable  time  and  the 
simple  reason  for  this  is  that  rules  of 
engagement  will  mandate  a  man-in-the-loop 
to  satisfy  the  political  constraints.  Some  will 
argue  that  this  man-in-the-loop  function  can 
be  achieved  remotely  but  they  have  yet  to 
prove  that  totally  secure,  reliable 
communications  can  be  achieved  world¬ 
wide,  under  hostile  conditions,  and 
throughout  the  24  hr  period.  Historically,  as 
there  have  been  major  leaps  in  technology, 
the  rules  of  engagement  have  been 
tightened.  Indeed,  before  the  cessation  of 
hostilities  in  Bosnia,  the  3  Star  General  who 
ran  the  air  operation  from  his  HQ  in  Italy 
would  often  talk  directly  to  the  pilot  about 
the  target  he  was  about  to  be  given 
permission  to  engage.  Some  may  say  this  is 
an  isolated  example  and  that  we  must 
provision  purely  for  a  NATO  Regional 
Conflict.  I  and  most  of  the  military 
disagree.  Since  1990 1  have  personally 
flown  on  operations  during  the  Gulf  War, 
policing  IRAQ  and  providing  Close  Air 
Support  to  the  UNPROFOR  in  Bosnia. 
None  of  those  operations  conform  to  the 
Regional  Conflict  model  and  throughout  the 
rules  of  engagement  were  tightened  thanks, 
primarily,  to  the  media.  We  the  military 
must  maintain  the  flexibility  to  operate 
anywhere  at  any  time  in  any  role  the 
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politicians  see  fit  to  employ  us.  Hexibility  is 
a  subject  to  which  I  will  return. 

So  if  you  accept  that,  for  the  foreseeable 
future  the  cockpit  will  be  manned  in  combat 
aircraft  what  form  will  it  take?  The  answer 
to  that  one  is  simple  - 1  don’t  know  and  I 
leave  it  to  the  aerospace  companies  and 
research  scientists  to  give  me  (being  the 
MOD)  that  knowledge.  I  do  know, 
however,  what  I  wish  that  cockpit  to  achieve 
and  that  is  that  it  must  allow  nie,  the  pilot,  to 
effectively  operate  that  aircraft,  throughout 
the  performance  envelope,  in  all  declared 
roles  making  use  of  all  available  information 
and  fully  integrated  within  the 
architecture.  In  addition,  I  must  be  able  to 
carry  out  the  intentions  of  the  political 
leaders,  military  commanders  and,  some 
would  argue,  myself.  A  tall  order  I  hear  you 
cry.  Let  me  try  and  define  further. 

Should  the  cockpits  have  1  or  2  seats?  This 
could  be  the  subject  of  a  whole  workshop  in 
itself  and  probably  a  very  emotive  one. 
However,  I  believe  that  cost  alone  will 
dictate  that  future  aircraft  will  be  single-seat. 
Indeed,  we  have  already  gone  down  that 
route  in  Europe  with  the  procurement  of 
Eurofighter  and  our  cousins  in  the  US  are 
conunitted  to  single  seat  with  F22  and  JSF. 
There  is  no  doubt  that  current  platforms  with 
complicated  systems  need  a  2  seat  crew  to 
enable  them  to  operate  effectively  and, 
indeed,  the  argument  for  2  pairs  of  eyes 
versus  one  is  well  made.  Automation  will 
discharge  this  requirement,  providing  it  also 
adequately  maintains  situation  awareness, 
and  will  also  remove  the  problem  of  poor 
crew  co-operation,  a  factor  often  cited  in 
recent  accident  reports;  the  human  is  often 
lacking  in  communicating  with  his  own 
kind.  This  leaves  only  one  interface  -  the 
single  crew  member  and  the  avionics  -  an 
area  on  which  we  in  the  UK  plan  to  focus 
more  over  the  next  few  years. 

What  do  I  require  from  that  interface? 
Experience  has  taught  us  that  giving  too 
much  control  to  a  machine  can  be  fatal  -  the 
pilot  of  an  A320  attempting  a  fly-by  at  an 


airshow  found  that  out  when  the  aircraft 
would  not  allow  him  to  apply  power  because 
he  had  selected  a  landing  mode.  It  is  widely 
accepted,  although  there  are  still 
considerable  pockets  of  resistance,  that  man 
and  the  automation  must  work  as  a  team, 
particularly  whilst  that  man  is  responsible 
for  the  conduct  of  the  mission.  Moreover,  if 
the  pilot  is  to  maintain  control  then  he  must 
understand  and  indeed  trust  his  R2D2  -  a 
crucial  point.  We  must  avoid  an  often- 
uttered  conunent  on  flight  decks  and  in  front 
of  PCs  (particularly  mine)  -  “what’s  it  doing 
now”  -  if  that  teaming  is  to  work. 

What  level  of  automation  should  be  off¬ 
loaded  to  the  machine  and  what  should  rest 
with  the  man?  This  is  perhaps  the  crux  of 
the  matter.  I  believe  the  answer  is  that, 
ideally,  it  should  vary  depending  on  a 
multitude  of  factors.  Pilots  are  often  quoted 
as  saying  that  when  the  gear  comes  up  they 
lose  50%  of  their  brain  capacity  and  that 
when  the  come  under  fire  the  other  50% 
deserts  them.  I  remember  experiencing 
something  along  those  lines  on  Day  1  of  the 
Gulf  War  in  my  Jaguar,  an  aircraft  not  noted 
for  its  automation.  However,  by  Day  10  the 
experience  of  the  previous  missions  had 
been  assimilated  and  my  spare  capacity  had 
improved  immeasurably.  I  would  have  been 
enormously  grateful  if,  on  Day  1  automation 
could  have  off-loaded  all  but  the  crucial  task 
of  locating  and  engaging  the  target.  By  Day 
10 1  would  have  been  happy  to  take  back 
more  of  the  tasks  such  as  threat  avoidance, 
navigation  and  tactical  flying  to  absorb  the 
spare  capacity.  Heaven  forbid  a  pilot  should 
be  bored  or  feel  he  lacks  control  in  the  heat 
of  the  battle  -  it  would  do  his  mental  state  no 
good  at  all!  To  be  serious  though,  the  level 
of  support  should  depend  on  his  spare 
capacity  and  that  capacity  is  often  dependent 
on  his  experience  level.  How  this 
interaction  is  achieved  is  the  difficult  exam 
question  but  should  result  in  a  form  of 
“adaptive  automated  decision  support”. 

Are  there  any  capabilities  that  electronic 
crewmember  will  be  unable  to  achieve?  I 
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believe  the  answer  to  be  yes  and  this  returns 
to  my  previous  reference  to  flexibility.  As 
far  as  I’m  aware  computers  cannot  replicate 
the  ingenuity  of  the  human  mind  and  this  is 
where  the  human  flexibility  wins  over.  The 
human  can  deal  with  situations  which  are 
totally  new  or  unplanned,  quickly  evaluate 
the  scenario  and  chart  a  course  through 
unfamiliar  territory  -  he  may  call  on 
previous  experience  but  he  may  also  rely  on 
innovation,  intuition,  imagination  and 
visualisation.  The  electronic  crewmember 
will,  at  the  moment,  lack  these  qualities  and 
thus  flexibility.  A  question  that  I  have  been 
asked  is  “can  you  train  for  or  leam 
flexibility”?  Simply  the  answer  is  no.  I 
spent  3  years  instracting  at  our  fast  jet 
training  school  and  you  could  very  quickly 
ascertain  if  a  student  had  the  make-up  to 
succeed.  The  flexible  student  quickly 
assimilated  the  requirements  and  could  cope 
with  most  everything  you  threw  at  him.  The 
rigid  student  had  to  see  something  first 
before  he  could  successfully  negotiate  the 
problem.  You  cannot  teach  that  flexibility 
(or  at  least  I  couldn’t). 

Another  capability  which  the  electronic 
crewmember  lacks  is  the  ability  to  take 
calculated  risks.  In  air-to-air  combat  2 
opponents  may  be  equally  capable  and  it  is 
often  the  one  who  makes  the  first  mistake 
who  loses.  This  may  add  fuel  to  the 
argument  that  the  machine  should  fly  the 
combat  as  it  will  not  make  the  mistake. 
However,  it  is  often  the  pilot  who  takes  the 
risk  which  wins  the  engagement.  A  World 
War  2  Battle  of  Britain  commander  is 
reputed  to  have  said  “don’t  send  me  good 
squadron  commanders,  send  me  lucky 
ones”.  I  believe  that  you  often  make  your 
own  luck  and  there  are  many  highly 
decorated  military  men  who  turned  the  tide 
of  a  battle  or  an  engagement  by  taking  a 
risk.  The  machine  will  not  take  that  risk! 

In  addition,  I  also  believe  that,  given  certain 
conditions,  the  human  can  fly  an  aircraft 
lower  than  the  automatics  can.  The  lowest 
setting  that  the  Tornado  Terrain  Following 
Radar  can  be  set  for  automatic  operation  is 


200  feet.  A  reconnaissance  Tornado  was 
tasked  during  the  Gulf  War  to  go  and  find 
the  Republican  Guard  in  the  Wadi  al  Batin. 
The  weather  and  the  aircraft  sensor 
capability  dictated  that  the  only  way  to 
achieve  the  mission  was  at  low  level.  As  it 
was  perceived  to  be  a  high  threat  area  the 
pilot  elected  to  fly  the  mission  manually 
using  the  radar  information  at  100  feet  -  the 
mission  was  a  success.  The  issue  of  flight 
safety  criticality  will  always  limit  the 
clearance  an  aircraft  can  achieve  when 
flying  close  to  the  ground.  We  in  the  Jaguar 
Force  trained  at  30  feet  over  the  desert  - 
thankfully  we  never  needed  it!  Could  a 
machine  have  achieved  that  even  with 
intelligent  vision! 

So  to  conclude.  Provided  that  we  approach 
the  problem  in  a  reasoned  manner  the 
human  electronic  crew  is  the  right  stuff.  It 
must,  however,  use  the  best  assets  of  each 
element,  allowing  for  the  situation  that  the 
pilot  finds  himself  in.  The  requirement  is 
not  easy  to  achieve  but  we  should  strive  to 
achieve  it.  So  the  real  question  is  what  mix 
is  the  right  stuff  -  the  theme  of  the 
workshop. 

I  hope  I  have  given  you  something  to  think 
about  -  if  not  please  humour  me  and  buy  me 
a  drink  later.  I  wish  you  all  an  enjoyable 
and  fruitful  workshop. 
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1.  SUMMARY 

In  order  for  the  human-electronic  interface  to  function 
effectively,  the  electronic  crewmember,  as  well  as  all 
electronic  interfaces  must  be  designed  in  accordance 
with  human  computer  interface  (HCI)  principles  and 
guidelines.  The  US  Department  of  Defense  (DOD)  has 
been  working  to  develop  Human  Computer  Interface 
(HCI)  guides,  handbooks,  and  standardization 
documents  as  part  of  their  interoperability  and  military 
standards  reform  initiatives.  Few  of  these  documents, 
however,  address  the  unique  HCI  requirements  of  the 
military  aviation  operational  environment.  A  need 
exists  for  a  set  of  guidelines  that  can  address  these 
unique  HCI  requirements  and  serve  as  a  resource  during 
the  cockpit  HCI  design  process.  The  Tri-Service 
Aviation  Human  Computer  Interface  (AHCI)  Style 
Guide  recently  developed  for  the  US  Joint  Cocl^it 
Office  will  fill  this  need.  This  paper  summarizes  the 
AHCI  Style  Guide  development  process  and  highlights 
the  design  guidelines  that  are  most  relevant  to  the 
design  of  the  electronic  crewmember  interface. 

2.  INTRODUCTION 

The  need  for  cockpit  HCI  design  guidance  stems  from 
the  rapid  development  of  software  and  hardware 
capabilities  within  the  avionics  industry,  including 
advances  in  computers,  sensors,  data  processing,  digital 
data  communications,  and  sophisticated  electronic 
display  formatting  techniques.  The  application  of  these 
capabilities  within  the  cockpit  has  led  to  (1)  a  profusion 
of  information  being  provided  to  the  crew  members  and 
(2)  the  increased  use  of  automated  systems  within  the 
cockpit.  A  significant  result  of  these  trends  is  an 
expansion  in  the  crew  member’s  role  from  an  active 


controller  who  directly  interacts  with  individual 
subsystems,  to  a  supervisory  controller  who  monitors 
and  manages  multiple  automated  subsystems  and 
information  sources. 

Electronic  interfaces  now  play  a  central  role  in  the 
modem  cockpit  interface,  providing  the  crew  with 
access  to  flight,  weapon,  navigation,  communications, 
defensive  systems,  offensive  systems,  and  other  system 
functions  and  information.  It  is  critical  that  these 
interfaces  are  designed  for  both  flexibility  and 
efficiency  to  support  a  wide  variety  of  crew  functions, 
and  to  allow  the  crews  to  effectively  use  them  in 
demanding,  time-critical  conditions.  A  flexible  and 
efficient  interface  design  would  also  enable  the 
crewmember  to  seamlessly  transition  from  one  task  to 
another,  as  required  by  different  mission  phases,  or  in 
response  to  a  highly  dynamic  combat  environment.  In 
short,  the  cockpit  HCI  should  be  a  natural,  intuitive 
interface  that  supplements  and  compliments  the 
crewmember’s  own  capabilities,  procedures  and 
techniques.  A  source  of  design  guidance  and  lessons 
learned  to  aid  designers  in  achieving  these  goals,  and  to 
aid  in  standardization  for  electronic  interfaces  across 
systems  is  needed.  The  Tri-Service  Aviation  Human 
Computer  Interface  (AHCI)  Style  Guide  is  being 
developed  to  fill  this  need. 

The  AHCI  style  guide  will  provide  guidance  for  HCI 
requirements  specific  to  the  military  aviation  domain. 
In  so  doing,  the  AHCI  style  guide  will  extend  existing 
DOD  HCI  style  guides  and  handbooks.  The  guidance 
provided  will  also  help  designers  comply  with  DOD 
interoperability,  intraoperability  and  open  systems 
architecture  objectives  and  standards  reform  initiatives. 
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3.  APPROACH 

3.1  Crew-Centered  Design  Philosophy 

The  AHCI  style  guide  was  developed  with  an  emphasis 
on  a  “crew-centered  design”  philosophy.  The  crew- 
centered  design  philosophy  encompasses  the  view  that 
effective  human-computer  interfaces  are  developed 
through  a  basic  understanding  of  the  functional  and 
information  requirements  of  the  operational  domain  as 
they  relate  to  the  crewmember’s  capabilities  and 
limitations.  This  understanding  is  critical  in  cockpit 
interface  design  because  it  enables  the  designer  to 
develop  an  interface  that  directly  supports  the  crew  in 
task  accomplishment,  maintaining  situation  awareness 
and  preserving  aircrew  safety. 

Following  this  philosophy,  an  integrated  systems  design 
approach  was  taken  in  the  development  of  the 
guidelines.  The  integrated  systems  approach  is  based 
on  the  notion  that  an  AHCI  typically  consists  of  a  suite 
of  interrelated  display  hardware,  programmable  display 
formats,  and  control  devices  that  are  linked  together 
through  dialogs  that  provide  the  method  by  which  the 
crew  accomplishes  “transactions”  with  the  system.  The 
approach  taken  for  the  AHCI  style  guide  emphasizes  the 
interaction  between  AHCI  system  components, 
including  the  crewmember  as  an  integral  component, 
and  the  need  for  an  integrated  AHCI  system  to  support 
the  diversity  and  multitude  of  tasks  that  the  crew  must 
accomplish  in  the  aviation  operational  environment. 

3.2  Operational  Considerations 

Prior  to  development  of  the  AHCI  style  guide,  the 
military  aviation  operational  environment  was  analyzed 
to  more  clearly  define  the  context  in  which  the  AHCI 
design  is  to  be  accomplished.  This  analysis  identified 
several  characteristics  that  are  common,  to  varying 
degrees,  across  a  broad  range  of  military  platforms  and 
missions,  and  that  can  have  a  significant  influence  on 
aircrew  performance  and  AHCI  design.  These  include: 

•  Information  Intensity.  Advances  in  computer  and 
sensor  technology,  data  processing,  and  digital  data 
communications  have  resulted  in  an  enormous  amount 
of  information  available  to  the  crew  in  modem  military 
cockpits.  The  multitude  of  information  currently 
available,  compounded  with  recent  trends  in  reducing 
cockpit  crew  size  has  created  the  very  real  potential  for 
information  overload  on  the  remaining  crewmembers. 

•  Multi-Tasking.  Crewmembers  on  all  aircraft 
platforms  are  frequently  required  to  perform  multiple 
tasks  simultaneously  and  to  attend  to  information 
presented  from  multiple  sources. 


•  Time  Constrained.  Many  missions  must  be 
performed  and  key  decisions  must  be  made  within  strict 
time  constraints. 

•  Severe  Consequence  for  Error.  In  many  missions, 
especially  combat  missions,  there  are  mission  critical 
functions  that  may  impose  severe  consequences  for 
crewmember  error. 

•  Dynamic  Environment.  Many  mission  can  be 
characterized  by  a  dynamic  environment  that  requires 
the  crewmember  to  be  responsive  to  real-time  changes 
in  tasking  and  the  mission  environment. 

•  High  Workload.  The  military  aviation  environment 
can  often  be  characterized  by  high  task  workload  and 
psychological  stress.  This  is  more  evident  in  combat 
missions,  during  critical  phases  of  flight,  and  may 
become  more  prevalent  with  ongoing  efforts  to  reduce 
crew  size. 

•  Physical  Constraints.  The  cockpit  environment 
imposes  various  physical  constraints  on  the 
crewmembers  that  may  impair  their  ability  to 
adequately  observe  all  system  displays  and  reach  all 
controls.  The  use  of  night  vision  goggles,  laser  eye 
protection,  and  nuclear,  biological  and  chemical  (NBC) 
gear  may  contribute  to  these  physical  constraints.  The 
ambient  noise  and  lighting  conditions  found  in  some 
cockpits  may  also  affect  the  performance  of  certain 
tasks. 

These  characteristics  were  taken  into  consideration  in 
the  development  of  the  AHCI  design  goals  that  form  the 
foundation  of  the  design  guidelines  presented  in  the 
AHCI  style  guide. 

4.  DESIGN  GOALS 

The  AHCI  design  goals  provide  a  link  between  the 
aviation  operational  environment  and  the  guidance 
provided  in  the  AHCI  style  guide.  The  guidelines  were 
developed  with  these  overall  goals  in  mind.  Taken  as  a 
whole,  the  design  goals  provide  a  framework  for 
interface  design  through  the  application  of  the  AHCI 
guidelines.  The  design  goals  include  the  following. 

•  Maximize  Situation  Awareness  (SA).  The  AHCI 
should  be  designed  to  facilitate  crewmember  awareness 
of  both  external  and  internal  events  as  they  relate  to  the 
cockpit  tasks.  Evidence  is  accumulating  that  the  loss  of 
SA  is  a  limiting  factor  on  mission  effectiveness  and 
plays  a  significant  role  in  aircraft  accidents  and 
incidents.  The  ability  to  anticipate  events  and  to  plan 
reactively  can  ultimately  determine  the  success  of  a 
mission.  One  of  the  first  things  that  can  be  done  to  help 
pilots  achieve  SA  in  a  demanding  environment  is  to 
improve  the  pilot  vehicle  interface  so  that  the  required 


6 


information  can  be  gleaned  with  a  minimum  amount  of 
workload.  (Endsley,  1) 

•  Minimize  Workload,  The  interface  should  be 
designed  so  that  the  crewmember  can  efficiently 
accomplish  all  functions  required  to  achieve  the  mission 
objectives  without  exceeding  his/her  cogmtive, 
perceptual  or  physical  capabilities.  Unnecessary 
workload  may  be  required  to  find  information  in  a 
multitude  of  screens  available,  acquire  information  from 
a  given  display  format  while  filtering  out  unneeded  or 
competing  information,  and  to  integrate  or  interpolate 
data  in  the  form  needed  for  decision  making.  (1) 

•  Provide  Efficient  Access  to  Critical  Information. 
One  of  the  best  ways  to  improve  the  crewmember’s  SA 
and  reduce  workload  in  a  demanding  environment  is  to 
design  the  interface  to  provide  access  to  critical 
information  with  minimal  effort  required.  The  AHCI 
should  provide  task-relevant  information,  rather  than 
raw  data,  to  minimize  the  need  for  the  crewmember  to 
mentally  integrate  information  from  multiple  sources.  It 
should  also  allow  the  crewmember  to  quickly  locate  and 
extract  critical  information  in  response  to  changes  in  the 
mission  environment. 

•  Maximize  System  and  Mode  Awareness,  The 
interface  should  be  designed  to  facilitate  crewmember 
cognizance  of  the  aircraft  systems  and  automation  mode 
state.  As  aircraft  subsystems  become  more  automated, 
modes  have  been  increasingly  used  as  a  general  method 
of  human-automation  interaction.  One  characteristic  of 
modes  is  that  the  same  pilot  action  may  result  in 
different  system  behavior  depending  on  system  mode. 
Thus,  mode  confusion  can  result  when  the  pilot 
misinterprets  the  current  mode  state  or  the  transitioning 
from  one  mode  to  another.  Pilot  error  resulting  from 
mode  confusion  is  considered  to  be  a  significant 
contributor  to  accidents  involving  highly  automated 
aircraft.  (Degani,  2) 

•  Minimize  ** Head-Down''  Time.  To  the  extent 
possible,  the  AHCI  should  be  designed  for  “head-up” 
operation,  and  to  minimize  “head-down”  time.  Such  a 
design  will  enable  the  pilot  to  maintain  awareness  of 
critical  information  in  the  external  environment.  Overly 
complex  interfaces  can  draw  the  crewmember’s 
attention  “head-down”  and  can  result  in  fixation  on  a 
single  system  or  display,  distracting  the  crewmember 
from  attending  to  mission-critical  information. 

•  Conform  to  Hands-On"  Design  Philosophy.  A 
“hands-on”  design  philosophy  helps  to  improve  multi¬ 
tasking  in  a  high  workload  environment.  Locating 
frequently  needed  or  critical  switches  on  the  controls 
allows  the  crewmember  to  easily  access  the  controls 
while  performing  the  primary  task  of  flying  the  aircraft. 


Hands-on  control  allows  for  a  more  immediate  and 
effective  operation  during  the  most  critical  phases  of 
flight.  The  design  also  provides  a  more  direct  interface 
that  promotes  smooth  transitioning  among  various 
control  tasks  with  minimal  physical  and  mental  effort. 

•  Minimize  and  Effectively  Manage  Errors.  The 
interface  should  be  designed  to  reduce  the  potential  for 
error  and  facilitate  the  recovery  from  error  should  one 
occur.  Within  the  cockpit,  there  are  many  mission 
critical  functions  that  impose  severe  consequences  for 
error.  Even  when  the  error  is  not  critical,  the 
crewmember  must  determine  the  cause  of  the  problem 
and  how  to  recover  from  the  error.  This  may  require 
additional  head-down  time,  increase  crewmember 
workload,  and  detract  from  the  accomplishment  of  other 
mission  tasks.  To  effectively  manage  errors,  the 
interface  should  be  designed  to  1)  minimize  the  chance 
of  errors,  2)  minimize  the  consequence  of  errors,  3) 
promote  the  detection  of  errors  when  they  occur,  and  4) 
facilitate  the  correction  of  errors  once  they  are  detected. 
(Norman,  3) 

5.  AHCI  DESIGN  PRINCIPLES  AND 
GUIDELINES 

5.1  Guideline  Development  Process 

The  design  goals  served  as  the  framework  from  which 
the  guidelines  were  developed.  The  AHCI  guidelines 
were  derived  from  relevant  HCI  specifications  and 
standards  from  the  military  services  and  various 
commercial  establishments.  Principle  among  these 
documents  was  the  DOD  Technical  Architecture 
Framework  for  Information  Management  (TAFIM,  4). 
Where  existing  guidelines  were  not  available,  they  were 
generated  based  on  published  HCI  and  cockpit  research 
results.  These  guidelines  were  then  tailored  to  the 
specific  aviation  domain  taking  into  consideration  the 
typical  AHCI  components,  their  role  in  the  overall 
cockpit  interface,  and  the  aviation  operational 
environment. 

5.2  Guideline  Categories 

Understanding  that  the  AHCI  is  made  up  of  display 
hardware,  programmable  display  formats,  and  control 
devices,  the  style  guide  development  team  concluded 
that  it  was  necessary  to  provide  guidelines  on  designing 
each  component  of  the  AHCI.  The  style  guide  provides 
guidelines  for  typical  AHCI  components  and  also 
provides  guidelines  on  integrating  these  components 
with  each  other  and  into  the  cockpit  as  a  whole.  By 
providing  guidance  on  each  component  as  well  as  the 
integration  of  these  components,  the  AHCI  style  guide 
presents  HCI  and  human  factors  design  principles  as 
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they  apply  specifically  to  the  unique  problem  of 
aviation  cockpit  interface  design.  After  compiling  the 
guidelines  relevant  to  AHCI  design,  it  was  evident  that 
the  guidelines  fell  into  four  major  categories;  one 
category  that  encompasses  high-level,  system 
integration  issues,  and  three  categories,  each  dealing 
with  a  specific  component  of  the  AHCI.  The  AHCI 
guidelines  are  presented  in  the  following  categories: 

1)  Cockpit  Information  Management  guidelines  provide 
system-level  design  guidance  for  managing  cockpit 
information  for  integrated  AHCI  system  design.  These 
guidelines  were  developed  with  the  understanding  that 
the  integrated  cockpit  design,  as  a  whole,  is  greater  than 
the  sum  of  its  individual  component  designs.  Thus, 
cockpit  information  management  functions  are 
addressed  in  the  context  of  designing  an  integrated 
AHCI  that  will  facilitate  the  crewmember  in  task  and 
information  management,  and  in  acquiring  and 
maintaining  situational  awareness.  Guidelines  in  this 
category  address  several  topic  areas  including: 
consistent  and  predictable  interface;  system 
compatibility  for  task  management;  information 
presentation  and  integration;  display  management; 
designing  for  crew  coordination;  interacting  with 
automated  systems;  mode  management;  and  effective 
use  of  decision  aids. 

2)  Dialog  guidelines  provide  guidance  for  the  design  of 
dialogs  that  allow  the  crewmember  to  conununicate  and 
interact  with  the  aviation  information  systems.  A  dialog 
is  defined  as  a  structured  series  of  HCI  transactions 
(crewmember  actions  paired  with  system  responses). 
Typical  dialog  types  include  menus,  graphic  interaction, 
alphanumeric  data  entry,  and  direct  system  command. 

3)  Information  Presentation  and  Formatting  guidelines 
provide  guidance  for  the  display  of  information  on 
electronic  media.  Specific  issues  addressed  within  this 
category  include  display  coding,  textual  displays, 
auditory  formats,  and  graphical  and  integrated  formats. 

4)  Displays  and  Controls  guidelines  provide  guidance 
for  a  variety  of  control  and  display  devices  typically 
implemented  in  the  AHCI  including  hands-on  controls, 
cursor  devices,  pushbuttons,  keyboards,  multi-function 
displays,  helmet-mounted  displays,  and  head-up 
displays. 

53  Example  Guidelines 

The  context  of  this  paper  does  not  allow  for  the 
presentation  and  discussion  of  each  guideline  contained 
in  the  AHCI  style  guide.  However,  the  following 
example  guidelines  are  presented  to  give  the  reader  a 
sense  of  the  intent  of  the  AHCI  Style  Guide  and  the 


diversity  of  the  guidelines  that  are  applicable  to  cockpit 
human-computer  interface  design. 

5.3.1  Information  Management  Guidelines 

Consistency  Across  Functions  -  Where  possible,  the 
cockpit  interfaces  for  different  functions  should  be 
consistent  and  predictable  in  terms  of  system  and 
automation  logic,  format  schemes,  organizational 
schemes,  coding  schemes,  usage  of  symbols  and 
abbreviations,  control  responses,  interface  dialog 
mechanization  and  feedback. 

Display  Formatting  Consistency  -  Display  formats 
should  have  a  consistent  structure  and  layout  for  similar 
functions.  Identical  or  similar  types  of  information  or 
display  elements  should  be  displayed  in  a  consistent 
manner,  regardless  of  the  source  of  origin.  Information 
encoding  schemes  used  throughout  the  interface  should 
be  consistently  applied  to  identical  or  similar  types  of 
information. 

System  Compatibility  with  Task  Requirements  - 
Displays,  controls  and  dialogs  should  be  functionally 
compatible  >vith  the  crewmember  and  task 
requirements,  to  promote  an  efficient  means  of 
accessing  and  acting  upon  task-relevant  and  critical 
information.  As  a  minimum,  displays,  controls  and 
associated  dialogs  should  be:  1)  optimized  for  task 
requirements,  2)  consistent  with  crewmember 
conventions  and  expectations,  3)  facilitate  the 
crewmember’s  ability  to  quickly  switch  attention  across 
multiple  information  sources  and  4)  minimize  the 
potential  for  error  especially  when  under  time-stresses 
and  high  workload  conditions. 

Direct  Access  of  Information  -  Information  should  be 
presented  to  the  crewmember  in  such  a  way  as  to 
facilitate  the  crewmember’s  ability  to  detect  and 
interpret  information,  while  placing  minimum  demands 
on  the  crewmember’s  perceptual,  memory,  and 
cognitive  capabilities.  This  is  applicable  to  the 
selection  of  display  modality  (auditory,  visual,  tactile), 
the  specific  information  content  presented,  the  format  of 
information  presentation  within  the  modality,  and  the 
level  of  precision  at  which  the  information  is  presented. 

Direct  Access  of  Primary  Flight  and  Critical 
Information  -  As  a  minimum,  the  system  should 
continuously  inform  the  pilot  of:  1)  primary  flight 
information,  2)  essential  information  for  the 
accomplishment  of  the  mission  tasks  and  3)  currently 
selected  system,  subsystem  or  display/control  modes. 
Other  information  requirements  should  be  based  upon 
mission,  task  and  information  analyses. 
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Prominent  Display  of  Essential  Information  - 
Essential  information  required  for  task  accomplishment 
should  be  readily  available  and  predominately  displayed 
at  the  forefront  of  the  crewmember’s  awareness.  This 
includes  information  that  is  commonly  used,  frequently 
cross-checked,  or  time-critical.  Conversely,  the  amount 
of  non-essential  information  should  be  minimized. 

Task-Based  Organizational  Structure  -  Information, 
controls,  and  options  should  be  segregated  into 
functional  groups  and  organized  on  the  basis  of  some 
logical  principle.  Major  tasks  and  associated  display 
pages  should  provide  the  predominant  structure  for  the 
interface. 

Tailoring  Display  Options  -  The  crewmember  should 
be  given  some  capability  to  control  the  amount,  format 
and  complexity  of  displayed  information  to  meet  task 
requirements.  This  includes  designating  the  display  of 
interest,  defining  cockpit  display  configurations, 
selecting  formats,  and  defining  default  configurations 
during  system  set-up. 

5.3.1.1  Automation  Guidelines 

Resource  Allocation  -  An  automated  function  should  be 
easier  to  operate  than  the  manual  function  it  replaces. 
Repetitive  and  predictable  tasks  should  be  allocated  to 
the  computer.  Tasks  that  are  performed  in  an 
unpredictable  environment,  require  flexibility  and 
adaptability,  or  goal-setting  and  switching  should  be 
allocated  to  the  human. 

Automated  Output  -  As  a  minimum,  automated  systems 
should:  1)  provide  sufficient  information  to  keep  the 
crewmember  informed  of  its  operating  mode,  intent, 
function  and  output,  2)  inform  the  crewmember  of 
automation  failure,  3)  inform  the  crewmember  if 
potentially  unsafe  modes  are  manually  selected,  4)  not 
interfere  with  crew  manual  task  performance  and  5)  be 
designed  to  allow  for  manual  override. 

Disruption  of  Task  Performance  -  Ensure  that 
automation  does  not  disrupt  the  performance  of  the  task. 
Design  control  automation  to  be  of  the  most  help  during 
times  of  highest  workload  and  less  help  during  times  of 
lowest  workload. 

Automation  Delimited  in  Authority  -  Control 
automation  should  not  be  able  to  endanger  an  aircraft  or 
make  a  difficult  situation  worse.  It  should  not  be  able 
to  cause  an  overspeed,  a  stall,  or  contact  with  the 
ground  without  explicit  instructions  from  the  pilot. 

Crewmember  Involvement  -  Keep  the  crewmember 
involved  m  the  automated  operation  by  requiring  of 
them  meaningful  and  relevant  tasks,  regardless  of  the 
level  of  management  being  utilized  by  them.  (Note: 


High  levels  of  strategic  management  have  the  potential 
to  decrease  pilot  involvement  beyond  desirable  limits. 
Keeping  pilots  involved  may  require  less  automation 
rather  them  more,  but  involvement  is  critical  to  their 
ability  to  remain  in  command  of  an  operation  and 
reenter  the  loop  in  case  of  a  failure).  (Billings,  5) 

Control  Automation  Flexibility  -  Control  automation 
should  provide  the  human  operator  with  an  appropriate 
range  of  control  and  management  options. 

5*3.1.2  Mode  Management  Guidelines 

Mode  Awareness  -  The  AHCI  design  should  facilitate 
crewmember’s  awareness  of  current  mode  state  and  the 
potential  interactions  among  modes,  especially  during 
critical  points  or  phases  in  the  flight. 

Transitioning  Between  Modes  -  The  crewmember 
should  have  the  capability  to  easily  switch  between 
modes.  Features  and  functions  that  are  common 
between  display  modes  should  be  as  consistent  as 
possible. 

Mode  Indicator  -  Design  display/control  modes  to  be 
visually  distinctive  or  provide  some  visual  indication  to 
make  the  current  mode  easily  identifiable. 

5,3.1.3  Decision  Aids  Guidelines 

Use  of  Decision  Aids  -  The  designer  should  consider 
using  decision  aids  for  managing  system  complexity,  for 
assisting  the  crewmember  to  cope  with  information 
overload,  and  for  focusing  the  crewmember’s  attention. 
Decision  aids  may  also  be  useful  for  overcoming  the 
following  human  limitations:  a)  dealing  with 

uncertainty,  b)  overcoming  emotional  components  of 
decision  making,  c)  memory  and  information-retention 
problems,  and  d)  systematic  and  cognitive  biases. 

Meaningful  Patterns  -  A  decision  aid  should 
automatically  notify  the  crewmember  of  meaningful 
patterns  or  events.  A  decision  aid  should  predict  future 
data  based  on  historical  data  and  alert  the  crewmember 
when  it  predicts  a  future  problem. 

Data  for  Making  Judgments  -  Decision  aids  should 
provide  data  on  which  to  base  judgments  rather  than 
commands  that  the  crewmember  must  execute. 

Feasible  Alternatives  -  If  available,  a  decision  aid 
should  provide  several  feasible  alternatives  from  which 
the  crewmember  can  choose. 

Awareness  of  Input  Logic  -  Provide  the  crewmember 
with  an  understanding  of  the  decision  rules  and  logic 
that  the  decision  aid  algorithm  is  utilizing. 
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5.3.2  Dialog  Guidelines 

User^Paced  Sequence  Control  -  Allow  the 
crewmembers  to  pace  control  entries,  rather  than 
requiring  the  crewmembers  to  keep  pace  with  computer 
processing  or  external  events.  This  will  allow  control 
entries  to  be  made  in  accordance  with  the 
crewmember’s  current  needs  and  available  time. 

Feedback  for  Crewmember  Actions  -  Provide  the 
crewmember  with  feedback  for  menu  selections,  data 
entries,  graphical  selections,  control  actions,  direct 
system  commands,  or  other  input  to  indicate  that  the 
action  has  been  accepted  or  not  accepted.  For  every 
action  by  the  crewmember  there  should  be  some 
apparent  reaction  from  the  system. 

Error  Handling  and  Feedback  -  The  HCI  should  be 
designed  to  provide  simple  error-handling,  both  by 
system  error-checking  and  ease  in  correcting  an 
identified  error.  Ensure  that  the  system  provides  a  clear 
indication  and  explanation  of  error  conditions. 

Menu  Structure  Based  on  Task  Analysis  -  Design 
cockpit  menu  structure  and  organization  based  on  a 
mission-level  human  factors  engineering  task  analysis. 
Consider  organizing  menus  around  subsystems  or 
operational  modes,  with  each  subsystem  or  mode 
functionality  accessible  from  a  top-level  menu  option. 

•  Menu  Tree  Structure  Appropriate  to  Cockpit 
Application  -  Limit  the  levels  in  cockpit  menu 
applications  (3  levels  is  typically  recommended)  so 
that  the  menu  tree  structure  is  broad  and  shallow 
rather  than  narrow  and  deep.  Crewmembers  can 
easily  become  lost  in  deep  levels  especially  during 
critical  tasks  within  the  aviation  operational 
environment. 

•  Menu  Type  Distinction  -  When  different  menu 
types  are  used  within  the  same  cockpit  application, 
there  should  be  a  unique  indication  that  denotes  the 
type  of  menu  and  the  actions  required  of  the 
crewmember. 

Graphical  Element  Selection  -  Provide  the 
crewmember  with  a  means  for  designating  and  selecting 
displayed  graphic  elements.  Ensure  that  it  is  clear  to  the 
crewmember  which  graphical  elements  are  selectable 
and  how  the  elements  are  to  be  selected.  When  the 
element  is  selected,  ensure  that  it  moves  to  the 
foreground  to  guarantee  that  it  is  not  obscured. 

•  Element  Selection  Area  -  Ensure  the  selection  area 
for  graphical  elements  is  of  appropriate  size  to 
enable  the  crewmember  to  easily  select  the 
elements  under  all  operational  conditions.  The 
selection  area  must  be  large  enough  so  that  the 


crewmember  can  accurately  position  the  cursor 
under  operational  flight  conditions  such  as 
vibration  and  g’s. 

•  Selection  Sensitivity  to  Vibration  -  When  objects 
are  selected  using  a  pointing  device,  ensure  the 
selection  method  is  not  sensitive  to  the  inherent 
vibration  of  the  aircraft. 

•  Limit  Need  for  Multiple  Entry  Devices  -  Design 
the  graphical  interaction  tasks  so  that  control 
actions  as  well  as  element  selection  is 
accomplished  by  the  same  input  device  to  limit  the 
need  to  shift  from  one  entry  device  to  another. 

•  Confirming  Selection  -  For  most  graphic 
applications,  pointing  and  selection  should  be  a 
dual  action,  first  positioning  a  cursor  at  a  desired 
position,  and  then  confirming  that  position  to  the 
computer. 

5.3.3  Information  Presentation  and  Formatting 
Guidelines 

Layout/Organization  of  Information  -  The  layout  of 
information  within  a  page  should  follow  some  logical, 
organizing  principle  that  crewmembers  can  recognize 
and  apply.  Data  should  be  grouped  on  the  basis  of  this 
principle,  considering  the  trade-offs  derived  from  task 
analysis.  Groups  of  data  should  be  distinctively 
displayed  using  blank  space,  surrounding  lines  or 
different  intensity  levels  as  appropriate.  Ensure  that  the 
location  of  recurring  functional  groups  and  individual 
items  is  consistent  across  MFD  pages  and  displays. 

•  Integrated  Format  -  When  the  crewmember  is 
required  to  assess  the  relation  among  different  data 
elements,  the  data  should  be  provided  in  an 
integrated  format  rather  than  partitioning  them  into 
separate  windows  or  display  pages. 

•  Display  of  Trend  Information  -  Format  and 
display  information  to  enable  the  crewmember  to 
easily  acquire  historical  trend  information  for  those 
tasks  that  require  some  estimate  of  this  information 
for  task  accomplishment. 

Content  of  Display  Information  -  The  content  of 
displayed  information  should  be  that  which  is  best 
suited  for  the  crewmember  at  any  given  point  of  time. 

•  Information  Quantity  -  The  quantity  of 
information  displayed  to  a  crewmember  should  be 
limited  to  that  which  is  necessary  for  the 
performance  of  specific  tasks,  maintaining  situation 
awareness,  or  the  making  of  decisions. 

•  Information  Precision  -  The  precision  of  the 
information  displayed  should  be  at  the  level 
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necessary  for  the  accomplishment  of  crewmember 
actions  or  decisions. 

Usable  Form  Consistent  with  Crew  Conventions  - 
Display  data  in  the  most  directly  usable  from  so  that  the 
crewmember  is  not  required  to  transpose,  compute, 
interpolate,  or  mentally  translate  data  into  usable  units, 
or  number  bases.  Display  information  only  as  accurately 
as  the  crewmember’s  decision  and  control  actions 
require.  Also  display  data  in  a  format  that  is  consistent 
with  crewmember  conventions  and  that  uses  familiar 
wording  or  task-oriented  j  argon. 

Display  Clutter  -  Displays  should  be  as  uncluttered  as 
possible.  Information  should  be  presented  simply  and 
in  a  well-organized  manner  and  the  display  should 
appear  clutter  free. 

•  Display  Density  -  Display  density  should  not 
exceed  50%,  and  preferably  should  be  less  than 
25%.  Density  should  be  minimized  for  displays  of 
critical  information.  The  unused  area  of  a  display 
page  should  be  distributed  to  separate  logical 
groups  and  categories  of  data. 

•  Prioritizing  Critical  and  Non^Critical  Information 
-  Minimize  the  information  density  on  a  display  by 
prioritizing  and  limiting  information  to  be 
displayed.  Prioritize  the  information  so  that  the 
most  important  or  critical  information  is  displayed 
at  all  times  and  less  important  or  critical 
information  can  be  displayed  upon  crewmember 
request. 

•  Display  of  Unnecessary/Old  Data  -  To  help 
alleviate  display  clutter,  unnecessary  borders 
should  not  be  used  on  the  display.  Scales  should 
not  be  cluttered  with  more  marks  than  necessary  for 
precision.  Old  data  points  should  be  removed  after 
some  fixed  period  of  time,  and  interim  information 
should  be  removed  from  the  display  once  it  is  no 
longer  needed. 

•  Declutter  Scheme  -  The  designer  should  develop  a 
hierarchy  of  information  needs  and  incorporate  this 
hierarchy  into  a  compatible  declutter  capability. 


•  Crewmember  Capability  to  Declutter  -  Whenever 
possible,  provide  the  crewmember  with  a  means  for 
reducing  clutter  while  preserving  essential 
information. 

6.  CONCLUSION 

The  above  guidelines  give  the  reader  a  brief,  high-level 
representation  of  the  type  of  design  guidance  provided 
in  the  AHCI  Style  Guide.  Taken  as  a  whole,  the 
guidelines  contained  in  the  style  guide  will  promote 
development  of  an  integrated  cockpit  interface  that 
effectively  supports  the  crewmember’s  role  as  system, 
task  and  information  manager  within  modem  military 
aircraft.  A  draft  version  of  the  style  guide  is  currently 
being  reviewed  by  tri-service  representatives,  as  well  as 
representatives  from  academia  and  industry.  It  is  hoped 
that  once  approved,  this  document  will  become  widely 
accepted  as  a  useful  tool  in  the  development  of  modem 
cockpit  interfaces. 
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Abstract 

This  paper  describes  the  range  and  scope  of 
laboratory  prototype  Knowledge  Based  Decision 
Support  systems  for  Maritime  Air  applications 
conducted  by  the  KBS  Group  at  DERA  Famborough 
UK.  The  rationale  behind  the  choice  and 
development  of  the  requisite  tools  and  software  are 
described.  The  applications  include  Decision  Support 
for  Anti  Submarine  Warfare  (ASW),  and  Anti  Surface 
Warfare  (ASuW)  together  with  an  ASW/ASuW 
Technology  Demonstrator. 

1.0  Introduction 

1.1  KBS  Group  Background 

The  Knowledge  Based  Systems  Group  within 
Systems  Integration  Department  at  the  Defence 
Evaluation  and  Research  Agency  (DERA)  in 
Famborough  have  been  engaged  for  more  than  a 
decade  in  research  and  applications  of  knowledge 
based  decision  support.  (Ref  1)  The  initial  impetus 
was  the  general  search  for  a  means  of  managing 
aircrew  workload.  Mission  systems  were  being 
specified  for  airborne  use  where  manufacturers  claims 
for  improvement  in  mission  performance  were  high 
but  without  corresponding  assessment  of  the  role  of 
the  aircrew  required  to  attain  such  performance 
levels.  From  earlier  work  by  the  core  team  members 
on  less  sophisticated  airborne  systems  the  indications 
were  that  the  newer  proposed  mission  systems  would 
generate  a  wake  of  different,  additional  attentional 
demands.  The  emergent  technology  of  expert  systems 
was  then  considered  as  a  potentially  useful  avenue  to 
explore. 

1.2  Shells  and  Limitations 

The  then  new  LISP  based  Expert  Systems  shells  such 
as  Inference  Art  and  Intelicorps  KEE,  together  with 
lesser  known  proprietary  products  were  acquired  and 
experience  gained  in  diagnostic  level  problems  such 
as  replicating  aircraft  warning  panels.  Experience  in 


devising  and  using  structured  interview  techniques  in 
problem  identification  and  assessing  performance  in 
Human  Factors  studies  where  more  reliable  measures 
were  not  available  enabled  the  team  to  build  skeletal 
laboratory  demonstrators  when  coupled  with  the 
commercially  available  shells.  Whilst  the  early 
experiences  with  the  shells  indicated  the  potential  of 
the  approach  in  functionally  representing  the  required 
salient  features  in  narrowly  focused  airborne  domains 
the  software  speed  limitations  rapidly  became 
apparent. 

1.3  Application  Expansion 

It  was  realised  that  the  fundamental  design  of  LISP 
posed  an  impediment  to  the  real  time  demands  of 
airborne  applications.  To  extend  the  laboratory 
demonstrators  to  capability  which  would  interest 
military  customers  would  require  faster  software  and 
prototype  build  methods  far  beyond  those 
commercially  available.  The  group  proposed  that 
more  systematic  methods  of  knowledge  acquisition,  a 
more  appropriate  form  of  validation  methodology 
were  needed  for  Knowledge  Based  Systems  (KBS) 
development  together  with  a  real  time  capability  more 
in  line  with  airborne  requirements.  This  would 
require  considerable  research  investment,  but 
coincidentally,  the  United  States  Government  General 
Accounting  Office  (GOA)  report  published  in  1981 
drew  attention  to  financial  implications  of  continually 
assuming  that  the  increasing  significant  elements  that 
system  designers  were  unable  to  specify  in  complex 
military  systems  could  be  compensated  for  by  the 
ingenuity  of  human  operators. 

1.4  Realisation 

Following  the  publication  of  the  GAO  report,  a 
NATO  Working  Group  was  set  up  and  reported  in 
1984  that  KBS  should  be  examined  as  one  possible 
means  of  addressing  such  problems.  (Ref  2)  One  of 
the  authors  was  a  member  of  the  NATO  Working 
Group  and  used  the  rationale  for  the  need  for  research 
funding  that  if  expertise  was  needed  to  generate 
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complex  systems,  if  that  expertise  could  be 
incorporated  in  computer  code  within  the  mission 
systems  as  advice  then  overall  mission  performance 
ought  to  improve.  Such  arguments  were  successful 
and  paved  the  way  for  the  large  scale  UK-MOD 
fund^  research  programme  covering  KA  methods 
(PC  PACK)  (Ref  3)  real  time  software  (MUSE  and 
D-MUSE)  (Ref  4)  and  a  validation  methodology 
(VORTEX)  (Ref  5)  which  currently  are  being  applied 
to  demonstrators  for  knowledge  based  decision 
support  for  Maritime  Air  applications. 

2.0  The  First  Maritime  Air  Application  -  Anti 
Submarine  Warfare  (ASW) 

Previous  background  by  the  author  in  human  factors 
assessments  of  Maritime  Air  sensor  and  mission 
systems  led  to  an  awareness  that  the  task  was 
characterised  by  a  developing  situation  where  fine 
judgements  were  examined  rather  than  the  more 
deterministic  reactions  encountered  in  strike  mission 
management.  When  an  application  was  needed  to 
evaluate  the  capability  of  the  validation  methodology 
research  which  recommended  a  spiral  development 
life  cycle  for  rapid  prototypes  for  KBS  workstation 
demonstrators  then  the  developing  nature  of  the 
maritime  mission  was  seen  as  an  appropriate 
application  to  demonstrate  the  concept  of  KBS.  (Ref 
6)  The  skeletal  Anti-  Submarine  Warfare  (ASW) 
scenario  used  proved  doubly  effective  in 
demonstrating  the  tools  designed  for  validating  the 
knowledge  base  and  when  combined  with  the 
measures  of  effectiveness  defined  by  the  maritime 
customer  illustrated  that  the  tactical  advice  offered 
exceeded  the  customer  expectation  of  the  laboratory 
demonstrator.  (Ref  7)  The  Validation  of  Real  Time 
Expert  Systems  (VORTEX)  application  had  been 
written  in  LISP  due  to  the  groups  experience  with 
LISP  based  shells.  At  that  stage  it  would  have  been 
difficult  to  make  the  case  for  funding  a  separate  line 
of  software  development  due  to  the  large  scale  US- 
DARPA  investment  in  LISP  during  the  initial  phase 
of  the  Pilots  Associate  Programme.  (Ref  8)  However, 
the  Group’s  experience  in  sponsoring  the 
development  of  the  real  time  software  development 
toolkit  (MUSE)  and  its  successful  application  in  a 
laboratory  demonstration  of  a  multi  engine  helicopter 
warning  panel  and  its  performance  on  tapes  provided 
by  NASA  Ames  on  telemetered  systems  status  data 
from  the  X29  research  aircraft  provided  sufficient 
evidence  in  its  potential  to  secure  additional  funding. 
The  next  expanded  version  of  the  ASW  application 
which  focused  on  producing  a  decision  support 
system  for  controlling  more  than  one  platform  used 
MUSE  rather  than  LISP. 


3.0  The  Second  Maritime  Air  Application  -  Anti 
Surface  Warfare  (ASuW) 

At  the  same  time  that  the  group  was  developing  the 
LISP  funded  ASW  demonstrator  for  the  validation 
methodology  evaluation,  parallel  effort  was  also 
being  expended  in  applying  MUSE  to  a  skeletal 
mission  manager  workstation  demonstrator  using  a 
fixed  wing  strike  scenario.  (Ref  9)  Compared  with 
earlier  success  of  using  MUSE  with  diagnostic 
applications  with  multi  engine  helicopter  warning 
panel  and  the  NASA  X29  data  the  different  data  types 
and  varying  input  frequencies  began  to  reveal 
limitation  in  the  real  time  performance  of  the  MUSE 
software.  Additional  research  funding  was  then 
received  to  develop  a  multi  agent  real  time  capability 
(D-MUSE)  to  maintain  the  software  performance 
against  the  more  demanding  scenarios  envisaged. 
Maritime  air  experience  during  the  Gulf  War  had 
revealed  the  difficulties  in  tracking  high  speed  fast 
patrol  boats  which  would  also  camouflage  their 
presence  by  mooring  alongside  oil  rigs  or  inserting 
themselves  in  slow  moving  fishing  fleets.  This 
application  was  considered  ideal  to  assess  the  real 
time  capabilities  of  the  multi  agent  software  and  so  a 
knowledge  based  Anti  Surface  Warfare  (ASuW) 
decision  support  system  workstation  demonstrator 
was  built.  (Ref  10) 

4.0  Aircrew  Roles  in  ASW/ASuW  Aircraft 

Maritime  aircraft  are  required  to  operate  world-wide, 
often  at  short  notice  and  in  a  variety  of  roles.  To 
fulfil  such  demanding  requirements  the  aircraft  carry 
very  sophisticated  sensors  and  complex  systems 
which  are  configured  and  managed  by  the  aircrew  to 
most  effectively  meet  the  needs  of  the  varied 
missions.  Such  variation  in  operating  environments 
and  improved  capabilities  of  future  mission  systems 
led  to  the  increasingly  demanding  role  for  aircrew. 
This  role  is  characterised  by: 

1  A  need  to  adequately  consider  the  most 
appropriate  mode  in  which  to  operate  a  sensor  due  to 
the  increased  number  of  modes. 

2  The  need  to  handle  a  vastly  increased 
volume  of  data  provided  by  improvements  in  sensor 
performance.  This  is  generated  by  the  increased 
number  of  targets  likely  to  be  seen  over  longer 
ranges.  The  lower  signal  to  noise  ratios  at  which 
detection  is  possible  also  increases  the  number  of 
false  contacts. 
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3  Greater  uncertainties  being  generated 
regarding  track  identity,  position,  course  and  speed 
due  to  the  difficulties  in  classifying  and  localising 
targets  at  longer  ranges. 

4  Data  from  different  sensors  needs  therefore 
to  be  combined  in  order  to  improve  the  confidence  in 
track  identity,  position,  course  and  speed.  This 
requirement  to  use  sensors  co-operatively, 
dynamically  reviewing  the  combination,  create  a 
significant  challenge  to  aircrew. 

5  Reduction  in  contact  time  for  sensors  results 
from  continuing  improvement  in  threat  performance 
so  that  the  window  of  opportunity  for  aircrew  to 
detect,  recognise  and  react  to  new  contacts  is 
diminishing. 

The  increasingly  demanding  role  imposed  on  the 
aircrew  and  the  associated  time  criticality  associated 
with  the  necessary  decision  making  based  on 
assimilation,  integration  and  interpretation  of  data 
from  a  multi-sensor  mission  system  therefore  lends 
itself  to  a  knowledge  based  decision  support  solution. 

For  the  Anti  Submarine  Role  (ASW),  the  aircrew 
tasks  (UK  Observer,  US  Tacco)  which  could  be 
addressed  by  applying  KBS  technology  would  be: 

1  Deployment  of  Active  Dipping  Sonar  and 
sonobuoy  screening  barriers 

2  Active  and  passive  location 

3  Attack  and  re-attack 

4  Lost  contact  procedures 

5  Management  of  assets 

Similarly  in  the  Anti  Surface  Warfare  role  (ASuW) 
the  aircrew  tasks  would  be: 

1  Classification  of  surface  sensor  data 

2  Generation  of  surface  picture  based  on 
classification 

3  Path  predicted  for  associated  tracks 

4  Plan  area  search  routes 

5  Assign  contacts 

6  Route  production  for  confirmation  of 
identity  and  hostility  level  of  tracks 

5.0  The  ASW/ASuW  Technology  Demonstrator 
Programme  (TDP) 

5.1  Background  and  Rationale 

Other  departments  within  DERA,  particularly  those 
associated  with  airborne  and  submarine  surface 
sonars  and  anti  air  warfare  in  surface  ships  had  also 
been  examining  and  applying  expert  system 


technology.  The  Maritime  Air  Customer  decided  that 
a  Technology  Demonstrator  Programme  or  US/ATD 
be  mounted  as  a  risk  reduction  exercise  before 
considering  the  exploitation  of  Knowledge  Based 
Decision  Support  technology  in  a  mission  system  for 
the  next  generation  of  ASW/ASuW  airborne 
platforms.  To  test  the  need  for  such  decision  support 
and  to  establish  the  breadth  of  functionality  required 
structured  interviews  were  conducted  with 
authoritative  sources  of  future  operational 
requirements  for  maritime  airborne  platforms,  DERA 
research  sites  and  contractors  engaged  in  developing 
workstation  demonstrators.  A  functionality  matrix 
was  used  incorporating  a  weighting  schedule  agreed 
by  interview  participants  to  establish  need  for  and  the 
functionality  demanded  of  a  decision  support  system 
to  manage  workload,  maximise  the  use  of  mission 
systems  and  sensor  resources  to  achieve  consistency 
of  mission  performance. 

5.2  Organisation  and  Components 

Having  agreed  the  need  and  functionality  required  for 
ASW/ASuW  decision  support  the  many  workstation 
demonstrators  developed  under  the  sponsorship  of 
DERA  but  targeted  at  specific  areas  of  functionality 
were  assessed  for  their  relevance  to  the  declared  aim 
of  satisfying  the  Maritime  Air  Customer  requirement. 
A  rainbow  consortium  of  contractors  had  been 
formed  in  order  to  manage  the  intellectual  property 
rights  aspects  and  exploit  the  specialist  experience  of 
teams  engaged  in  the  wide  range  of  small  scale 
laboratory  demonstrators.  Representative  threat 
scenarios  were  provided  by  the  maritime  Air 
Customer.  The  Rainbow  Consortium  included 
contractors  with  experience  of  building  workstation 
demonstrators  in  the  target  domain,  simulation 
environments  to  evaluate  such  demonstrators,  data 
fusion  systems  and  software  to  implement  such 
schemes.  These  building  blocks  under  consideration 
for  the  TDP  included  7  separate  workstation 
demonstrators,  3  simulation  facilities,  2  computer 
simulation  environments,  2  real  time  software  toolkits 
and  a  data  fusion  system.  Evaluation  of  building 
blocks  was  conducted  using  a  selection  strategy  based 
on  weighted  requirements  matrix  including  tasks 
defined  in  the  approved  scenarios  together  with 
functionality  requirements.  (Ref  11)  This  assessment 
having  been  achieved  allowed  the  optimum 
architecture  to  be  defined  together  with  the  associated 
components.  Examination  of  maturity  levels, 
flexibility  and  implementation  considerations  in 
association  with  a  cost/benefit  analysis  led  to  the 
selection  of  the  architecture  and  necessary 
components  to  achieve  the  core  decision  support 
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system  to  implement  the  desired  level  of  complexity 
to  influence  the  Maritime  Air  Customer  of  the 
potential  of  the  technology.  This  rigorous  evaluation 
and  trade  off  study  resulted  in  the  selection  of  an 
architecture  which  included: 

1  A  computer  simulation  environment,  hosted 
on  workstation  which  incorporated  ASW  and  ASuW 
modelling  capability. 

2  A  data  fusion/association  capability  based  on 
AAW  Frigate  TDP  and  ASW  mission  system. 

3  The  ASW  and  ASuW  decision  support 
laboratory  workstation  demonstrators. 

4  The  real  time,  multi  agent  software 
development  toolkit  D-MUSE  due  to  its  level  of 
maturity;  richer  selection  of  programming  strategies; 
represented  the  core  of  the  key  building  blocks  and 
cold  be  interfaced  to  the  simulation  environment. 

5.3  Complexity 

Further  additional  tasks  are  under  discussion  for 
inclusion  by  the  contractors  and  the  Maritime  Air 
Customer  in  order  to  increase  the  capability  of  the 
ASW/ASuW  Knowledge  Based  Decision  Support 
Technical  Demonstrator  Programme.  Man-in-the- 
loop  evaluations  are  planned  in  order  to  demonstrate 
military  worth  of  knowledge  based  DSS  for  the 
Maritime  Air  Customer.  It  can  be  seen  that  the  nature 
and  complexity  of  the  tasks  are  very  different  from 
those  usually  encountered  in  the  literature.  A  recent 
NATO  KBS  Working  Group  reported  that  KBS 
technology  was  sufficiently  mature  for  applications  in 
aeronautics  and  space  due  to  their  potential  having 
been  demonstrated  in  France,  Germany  and  the  USA 
in  diagnostic  and  planning  tasks  in  fielded  systems. 
(Ref  12)  More  multi-function  systems  incorporating 
complex  architectures  were  reported  as  being  between 
the  laboratory  and  fielded  systems.  The  Maritime  Air 
TDP  described  in  this  paper  represents  such  an 
interim  system. 

6.0  Situation  Assessment 

6.1  Significance 

A  crucial  element  of  the  TDP  and  one  which 
embodied  many  of  the  questions  associated  with  the 
search  for  progress  mentioned  in  the  workshop 
calling  notice  is  that  of  Situation  Assessment. 
Situation  Assessment  (SA)  is  a  process  where  an 
assessment  of  the  compiled  tactical  picture  is  made 
with  the  aim  of  extracting  information  that  can  help 
aid  the  decision  making  process.  In  the  Maritime 
environment  this  process  can  be  performed  by  naval 


personnel  at  all  levels:  an  operator  examining  a  radar 
display,  a  ship’s  Air  Warfare  Officer  assimilating 
information  about  aircraft  that  are  transiting  within 
sensor  range,  an  Airborne  Early  Warning  observer 
ascertaining  the  type  and  role  of  an  approaching 
aircraft.  The  importance  of  the  SA  process  has 
already  been  recognised  by  the  most  of  the  designers 
of  knowledge-based  applications  and  this  is  shown  in 
the  number  of  decision  support  systems  that  utilise 
elements  of  S  A. 

6.2  Difficulties 

One  of  the  greatest  problems  experienced  in 
designing  a  system  that  performs  SA  is  that  it  is  often 
difficult  to  obtain  an  objective  description  of  the 
tactical  picture.  When  a  group  of  domain  experts  are 
presented  with  a  tactical  picture  that  has  little 
historical  data  associated  with  it  they  will  very  often 
give  different,  but  not  entirely  dissimilar,  opinions  of 
what  they  are  seeing.  The  best  that  anybody  who 
performs  knowledge  acquisition  with  a  group  of 
experts  can  hope  for  in  the  absence  of  a  structured 
methodology  is  a  general  consensus  of  opinion.  This 
is  just  one  reason  why  it  is  often  difficult  to 
implement  large-scale  knowledge-based  SA  systems 
because  if  its  output  is  based  upon  the  consensus  of 
one  group  of  experts  then  more  often  than  not  there 
will  exist  another  group  of  domain  experts  who  will 
disagree  with  it.  A  solution  to  this  problem  is  to  use 
official  documentation  about  the  domain  and  couple  it 
with  the  knowledge  that  has  been  elicited  from  the 
domain  experts.  For  the  purposes  of  the  TDP  this  has 
been  official  ASW/ ASuW  tactic  manuals  known  as 
TACMAN.  The  accuracy  of  the  output  of  a  system 
carrying  out  SA  can  be  severely  hampered  by  a 
number  of  reasons,  such  as  an  incomplete  knowledge 
base  caused  by  inappropriate  methods  of  knowledge 
acquisition.  Even  if  the  knowledge  base  is  complete  it 
may  still  fail  to  correctly  assess  the  tactical  picture 
because  a  target  has  not  been  in  contact  for  very  long 
because  of  the  limitations  in  the  detection  range  of  the 
sensors,  or  the  type  of  data  that  the  SA  system 
requires  is  not  complete,  or  consistent,  and  the  system 
does  not  possess  the  reasoning  capabilities  or  the 
inherent  teowledge  necessary  to  overcome  this 
problem. 

6.3  Solutions 

The  main  reason  for  the  existence  of  the  latter 
problem  is  that  early  attempts  at  computer  based  SA 
had  been  rule-based  implementations  that  have 
concentrated  on  formulating  just  one  opinion  about 
one  particular  target  or  a  group  of  targets.  It  is 
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inherent  that  rule-based  reasoning  systems  are  fairly 
inflexible,  especially  when  there  is  incomplete  data, 
because  of  their  foundations  in  logic.  Despite  these 
problems  rule-based  systems  are  still  one  of  the  most 
popular  ways  of  implementing  knowledge  bases  due 
to  the  natural  way  of  representing  domain  knowledge 
and  for  their  legibility  to  domain  experts.  To 
overcome  the  problem  of  dealing  with  incomplete 
data  recent  approaches,  as  used  in  the  decision 
support  elements  of  the  TDP,  have  advocated  a  multi¬ 
agent  based  multiple  hypothesis  approach  when 
assessing  the  identity,  role  and  intentions  of  a  target 
whereby  the  agents  will  use  domain  knowledge  to 
create  a  hypothesis  and  communicate  it  to  other 
agents  in  the  system.  With  this  approach  the  SA 
system  will  perform  its  functions  in  a  manner  that  it 
not  too  dissimilar  to  that  of  the  crewman  executing 
the  same  task.  As  more  information  becomes 
available  about  a  particular  target,  or  group  of  targets, 
the  system’s  agents  can  create,  update  or  eliminate 
multiple  hypotheses  about  them.  This  is  the  same 
method  used  to  perform  battlefield  SA  in  the  US 
Army  Rotorcraft  Pilot’s  Associate  programme  (Ref 
13).  A  multi-agent  system  may  be  designed  to  use  a 
selection  criteria  to  assign  the  target  the  same  values 
as  the  highest  scoring  hypothesis.  Alternatively  the 
system  may  be  designed  to  repeat  the  elimination 
process  until  such  time  as  when  their  is  sufficient  data 
as  to  justify  the  existence  of  just  one  hypothesis 
which  is  then  used  as  output.  The  advantage  of  such 
systems  is  that  they  can  offer  alternative  hypotheses 
about  targets  which  may  allow  a  decision  support 
systems  to  calculate  more  than  one  solution  to  a 
problem  (Ref  10) 

6.4  Complexity 

One  of  the  biggest  problems  of  designing  a  system 
that  performs  SA  is  how  much  domain  knowledge 
should  such  a  system  possess.  Should  it  be  able  to 
produce  information  about  a  tactical  picture  that  is 
suited  to  one  or  more  tactical  decision  support  aids, 
or  should  it  be  designed  to  perform  an  assessment  and 
then  tailor  its  output  in  a  format  that  will  allow 
existing  and  future  tactical  decision  support  aids  to 
fully  exploit  it.  Essentially  this  is  a  question  of 
whether  the  SA  system’s  knowledge  should  be  task 
specific  or  more  genetically  based  upon  the  type  of 
information  that  naval  personnel  retain  about  the 
operating  environment.  With  the  task  specific 
approach,  the  knowledge  that  is  encapsulated  in  the 
decision  support  system  is  orientated  to  deal  with  all 
the  functions  that  need  to  be  performed  to  complete 
the  task  successfully.  A  generically  based  SA  system 
would  be  required  to  encapsulate  knowledge  not  just 


about  the  specific  tasks  that  are  required  to  be 
performed  but  also  experiential  knowledge  about  the 
operating  environment  that  would  allow  it  to  ensure 
the  information  that  it  was  formulating  about  the 
tactical  picture  was  complete  and  consistent  with  that 
environment.  An  example  of  such  experiential 
knowledge  would  be  to  allow  the  SA  system  to  know 
what  the  mission  objectives  were  of  the  host  platform 
as  well  knowledge  about  the  tactics  used  by  the 
known  enemy.  The  system  would  then  be  able  to 
reason  more  adequately  and  replicate  human  decision 
making  more  closely.  However  the  inadequacy  of 
acquisition  methods  to  construct  the  completeness 
required  would  appear  to  currently  limit  the  scope  of 
generically  based  SA  systems.  As  the  tasks  that  the 
TDP  is  to  perform  were  clearly  defined  through  an 
extensive  process  of  knowledge  acquisition  it  was 
decided  that  task  specific  SA  would  more  accurately 
represent  that  found  in  the  operating  environment  and 
so  was  included  as  part  of  the  tactical  decision 
support  systems. 

7.0  Conclusions 

In  a  recent  journal  article,  Hayes-Roth,  drawing  on 
two  decades  of  research  experience  for  DARPA, 
contended  that  much  has  been  achieved  using  AI 
techniques  in  an  incremental  fashion  in  relatively 
small  scale  exercises.  (Ref  14)  However  effort 
needed  to  be  concentrated  over  a  period  of  time  in 
specific  domains  to  demonstrate  potential  before 
adapting  and  transferring  solutions  to  new  situations. 
Context  adaptable  building  blocks,  re-useable 
knowledge  and  composite  architectures  for  multi  task 
systems  were  reconunended  as  necessary  constituents 
of  a  future  strategy  for  AI.  The  authors’  of  this  paper 
contention  would  be  that  in  concentrating  on 
Maritime  Air  ASW  (Ref  15),  ASuW,  (Ref  16),  AEW 
(Ref  17)  for  almost  a  decade,  and  in  developing  the 
necessary  support  software  tools  and  methodologies, 
together  with  the  re-use  aspects  as  discussed  by  Prof. 
Shadbolt,  the  KBS  Group  within  the  DERA 
organisation  in  the  UK  already  conform  to  most 
aspects  of  the  Hayes-Roth  paradigm.  New  ground  is 
also  being  broken  in  the  Maritime  Air  domain  as 
represented  at  this  workshop  in  the  papers  by 
Zancanato  and  Davies  on  architectures  and  designs 
for  Airborne  Early  Warning,  Prof.  Shadbolt  on 
Knowledge  re-use  and  updating,  and  Me  Cloud  on 
validation  and  certification.  A  recent  US  survey 
indicated  that  the  KBS  tool  and  consultancy  market 
within  North  America  was  $258  Million  but  reported 
that  the  activity  in  the  UK  remains  conservative  in 
development  and  deployment  of  expert  systems. 
Maritime  Air  is  certainly  an  exception  to  this 
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generalisation  but  awareness  is  no  doubt  limited  due 
to  exposure  being  confined  to  forums  such  as  this 
workshop. 
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SUMMARY 

Considerable  efforts  have  been  spent  on  the  devel¬ 
opment  of  intelligent  crew  support  systems  for  com¬ 
mercial  transport  aircraft.  This  paper  is  intended  to 
reflect  briefly  on  crew  assistance  systems  of  the  past 
and  to  describe  the  state-of-the-art  of  electronic 
crewmembers  for  commercial  transport  aircraft. 

Finally,  the  application  of  intelligent  crew  support  in 
commercial  aircraft  within  the  immediate  future  is 
being  discussed. 

1.  INTRODUCTION 

The  role  of  the  crew  in  today’s  automated  flight  deck  of 
commercial  transport  aircraft  is  being  reviewed  and 
discussed  each  time  an  incident  or  accident  cause  has  to 
be  attributed  to  ,4iuman  error“. 

In  the  past,  the  aerospace  community  tried  to  overcome 
the  limitations  of  the  human  as  a  real  time  operator  by 
increasing  automation.  This  approach,  however,  failed 
completely.  One  could  observe  a  change  in  the  type  of 
errors,  but  the  accidents  due  to  human  error  still 
account  for  some  75  %  of  the  overall  accidents. 

In  the  early  1980’s,  it  was  realised  that  an  automation 
of  more  and  more  pilot  tasks  can  not  be  the  solution  to 
the  aircraft  safety  problem.  The  focus  of  attention 
shifted  towards  the  division  of  functions  between  the 
human  crew  and  the  aircraft  systems. 

In  parallel,  a  new  scientific  discipline  arose  ,  the  so- 
called  Artificial  Intelligence.  And  soon  the  idea  of  an 
Electronic  Crewmember  with  human-like  knowledge 
and  extensive  reasoning  capabilities  in  order  to  solve 
problems  came  up. 

The  perspective  was  overwhelming  :  Having  an  on¬ 
board  computer  system  that  complements  the  human 
crew,  capable  of  solving  complex  decision  problems  in 


fractions  of  a  second,  not  suffering  from  exhaustion  in 
high-activity  flight  phases  and  boredom  in  low-activity 
parts  of  the  flight.  And,  finally,  a  tool  that  might  detect 
and  prevent  any  possible  pilot  error. 

This  paper  is  an  attempt  to  reflect  on  15  years  of 
research  and  development  work  on  intelligent  assistant 
systems  for  the  transport  aircraft  and  to  outline  the 
future  application  in  operational  systems. 

2.  HISTORY  OF  AUTOMATION 

The  basic  task  of  a  pilot  is  to  stabilise  the  aircraft  and  to 
steer  an  intended  flight  path  by  manual  operation  of  the 
primary  aircraft  controls,  i.e.  elevator,  aileron  and 
rudder.  Figure  1  depicts  this  basic  control  loop,  which 
gives  immediate  feedback  to  the  pilot  with  regard  to  the 
actions  taken. 


Figure  1  :  Basic  Aircraft  Control  Loop 

From  the  early  days  of  aviation,  aircraft  designers  tried 
to  reduce  the  pilot’s  workload  by  automation.  It  started 
with  simple  wing  leveller  systems  that  stabilised  the 
aircraft  around  its  longitudinal  axis,  and  led  to  a  first 
level  of  crew  assistance  in  the  so-called  „autopilot“, 
which  provide  automatic  capture  and  hold  of  flight  path 
parameters,  like  heading  or  altitude.  This  automation 
step,  as  outlined  in  figure  2,  separated  the  pilot  from 
direct  control  of  the  aircraft  and  cut  him  off  the  control 
force  feedback. 
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Figure  2  :  First  Level  of  Crew  Assistance 

The  next  logical  step  was  based  on  the  question  : 
Where  does  the  autopilot  input  come  from  ?  Who,  for 
example,  defines  the  intercept  heading  to  an  assigned 
track  ?  This  was  the  origin  of  „flight  guidance  systems^ 
that  feed  information  into  the  autopilot,  separating  the 
pilot  from  the  direct  control  by  two  layers  (figure  3). 


Figure  3  :  Second  Level  of  Crew  Assistance 

The  most  sophisticated  step  in  automation  for  crew 
assistance  was  the  „flight  management  system^  (FMS). 
Figure  4  shows  the  present  situation,  with  the  FMS  as 
additional  layer  between  crew  and  aircraft. 

Now  the  crew  can  program  a  complete  flight  plan  into 
the  FMS,  which  decomposes  the  information  and  feeds 
the  proper  inputs  into  the  flight  guidance  system,  which 
in  turn  controls  the  autopilot,  that  finally  flies  the 
aircraft. 


Figure  4  :  Third  Level  of  Crew  Assistance 

Now  and  then,  this  procedure  ends  in  a  mountain,  like 
the  Cali  accident  proves,  where  the  crew  programmed  a 
wrong  navaid  into  their  FMS.  They  never  became 
aware  of  their  mistake. 

One  logical  step  ahead  -  from  the  perspective  of 
automation  -  could  now  be  the  „electronic  crewmem¬ 
ber"  that  finally  eliminates  the  crew  from  the  flight  deck 
( figure  5 ). 

While  this  -  for  obvious  reasons  -  might  be  a  sensible 
approach  for  military  fighter  aircraft,  an  application  in 
commercial  aircraft  is  unthinkable. 

The  concept  of  an  electronic  crewmember  or  crew 
assistant  is  rather  that  of  an  intelligent  support  system 
that  complements  the  crew  in  order  to  make  the  best  use 
of  their  resources  and  capabilities. 

As  defined  by  Onken  [  1  ],  an  electronic  crewmember  ( 
or  „Crew  Assistant" )  has  the  following  functional 
requirements  : 

L  Within  the  presentation  of  the  full  picture  of  the 
flight  situation  it  must  be  ensured  that  the  attention 
of  the  cockpit  crew  is  guided  towards  the 
objectively  most  urgent  task  or  subtask  of  that 
situation. 
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Figure  5  :  Crew  Elimination 

2.  If  the  first  basic  requirement  is  met,  and  if  a 
situation  still  comes  up  with  overcharge  of  the 
cockpit  crew  ( in  planning  and  plan  execution),  then 
this  situation  has  to  be  transferred,  by  technical 
means,  into  a  situation  that  can  be  handled  by  the 
crew  in  a  normal  manner. 

From  these  requirements  one  can  easily  infer  that  an 

electronic  crewmember  must  be  characterised  by  : 

•  a  knowledge  base  representing  the  knowledge  of  the 
flight  domain  and  aircraft  operation  domain,  and 

•  a  model  of  the  crew’s  behaviour  and  resources,  and 

•  reasoning  capabilities  in  order  to  continuously 
assess  the  flight  situation  and  crew  behaviour,  and 

•  capabilities  to  put  tasks  or  functions  into  the  context 
of  the  current  flight  and  crew  situation  ant  to  carry 
them  out  autonomously. 


hints  to  important  situation  elements  and  alerts  with 
respect  to  the  situation  as  a  whole  fulfil  the  EC 
requirement  #1.  EC  generated  proposals  for  further 
actions  and  the  possibility  to  execute  them  on  pilot’s 
acknowledgement  meet  the  EC  requirement  #2. 


Figure  6  :  Co-operation  of  Crew  and  EC 


It  is  quite  obvious,  that  these  features  define  a  system 
totally  different  from  those  automated  assistance 
systems  outlined  before. 

The  division  of  tasks  between  the  human  crew  and  the 
electronic  crewmember  should  be  adaptive  according  to 
the  situation  and  should  take  into  account  the  available 
crew  resources.  Figure  6  outlines  the  basic  principle. 
Figure  6  indicates  that  an  EC  has  to  be  seen  as  a 
complement  to  the  pilot,  not  as  a  substitute.  In  each  step 
of  the  situation  assessment  and  decision  process  chain, 
the  EC  may  support  the  crew,  if  necessary.  Possible 


The  next  section  will  introduce  the  past  and  present 
efforts  on  electronic  crewmembers  for  transport  air- 
craft. 


3.  CREW  ASSISTANT  HISTORY  AND 
STATE-OF-THE-ART 

One  of  the  first  programs  dealing  with  electronic 
crewmembers  was  the  ,J^ilot’s  Associate^  of  DARPA, 
which  was  targeted  at  assisting  pilots  in  tactical  combat 
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aircraft  (see  [  7  ]).  Similarly,  the  French  „Copilote 
Electronique“  program  [  8  ]  aims  at  knowledge-based 
pilot  support  in  combat  aircraft. 

In  the  transport  aircraft  domain,  efforts  for  know-ledge- 
based  pilot  support  started  in  1986  with  the  ASPIO 
project  [  2  ]  of  the  Universitat  der  Bundeswehr 
Munchen  (UniBwM).  ASPIO  (Assistant  for  Single- 
Pilot  IFR  Operation)  was  characterised  by  the  following 
features : 

•  Speech  input  and  output  as  primary  means  of 
communication  with  the  pilot 

•  Decision  aid  module  for  flight  planning  using  fuzzy 
reasoning 

•  Representation  of  the  pertinent  aircraft  operation 
knowledge  by  use  of  scripts  and  productions 

•  Monitoring  of  the  actual  pilot  behaviour  and 
alerting  in  case  of  errors 

In  1990,  the  ASPIO  system  was  successfully  tested  in  a 
flight  simulator. 

Based  on  the  lessons  learned  of  the  ASPIO  project,  a 
joint  project  of  UniBwM  and  Daimler-Benz  Aerospace 
was  launched  in  1991.  Its  goal  was  to  develop  a  cockpit 
assistant  system  ( GASSY  )  for  intelligent  crew  support 
in  commercial  transport  aircraft  in  order  to  ensure  flight 
safety,  reduce  pilot  workload  and  improve  the  flight 
plan  efficiency  (see  [  1  ]  and 
[3]). 

This  was  achieved  by  GASSY  with  the  following  basic 
functions : 

•  Assessment  of  the  actual  situation  elements  based 
on  all  available  dynamic  data  of  the  aircraft  and 
information  uplinked  by  ATG 

•  Autonomous  elaboration  of  flight  plan  and  proce¬ 
dure  proposals 

•  Autonomous  evaluation  of  the  actual  flight  situation 
based  on  extensive  knowledge  on  the  crew 
behaviour,  crew  error  models  and  crew  resources 

The  basic  structure  of  GASSY  is  depicted  in  figure  7. 


Figure  7  :  Functional  Blocks  of  GASSY 

The  functional  blocks  shown  in  figure  7  were  realised 
as  separate  processes  on  a  Silicon  Graphics  platform. 
The  amount  of  source  code  finally  reached  more  than 
200.000  lines  of  „G“  code. 

After  an  extensive  test  phase  in  flight  simulators  of 
UniBwM  and  Dasa,  the  GASSY  prototype  was  inte¬ 
grated  in  the  flying  testbed  ATT  AS  of  the  German 
Aerospace  Research  Establishment  DLR  at  Braun¬ 
schweig.  The  system  was  validated  in  1994  by  airline 
pilots  in  typical  IFR  flights  to  Hamburg,  Hannover  and 
Frankfurt  (see  [  4  ]). 

The  GASSY  flight  trials  confirmed  that 

•  the  use  of  speech  input  in  the  noisy  flight  deck 
environment  is  possible 

•  the  knowledge  based  assessment  of  the  pilot  be¬ 
haviour  also  works  under  severe  real  time  condi¬ 
tions 

•  pilot  errors  can  be  detected  and  corrected  before 
impacting  the  aircraft’s  safety 

•  inflight  planning  activities  can  be  accelerated  for  the 
benefit  of  rapid  reactions  to  a  new  situation 

•  user  acceptance  for  this  kind  of  support  system  is 
ensured 
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In  1993,  parallel  to  the  very  successful  CASSY  ac¬ 
tivities,  the  German  MOD  started  a  research  and 
technology  development  project  called  „Crew  Assistant 
Military  Aircraft^  (CAMA)  [  5  ]  in  order  to  investigate 
the  use  of  an  intelligent  electronic  crewmember  in  the 
military  transport  application.  CAMA  enhances  the 
CASSY  functions  by : 

•  a  tactical  situation  assessment 

•  a  ground  collision  avoidance  algorithm 

•  inclusion  of  the  pilot’s  eye  movements  in  the  pilot 
behaviour  assessment 

•  computer  vision  with  respect  to  the  outside  world 

•  low  level  flight  planning 

A  CAMA  demonstrator  is  supposed  to  be  validated  in 
flight  in  1999. 

The  overall  success  of  the  CASSY  and  CAMA  projects 
leads  to  the  question,  how  far  the  next  logical  step,  the 
implementation  as  a  commercial-off-the-shelf  avionics 
product,  is  ahead. 

Before  an  attempt  to  answer  this  question  is  made,  the 
next  section  will  try  to  describe  the  near-term  air  traffic 
scenario,  that  will  significantly  affect  the  flight  deck 
environment  and  the  airlines’  equipment  policy. 

4.  FUTURE  AIR  TRAFFIC  SCENARIO 

The  air  traffic  scenario  will  encounter  significant 
changes  during  the  next  decade.  Following  ICAO’s 
FANS  concept  (,f  uture  Air  Navigation  System“)  (see  [ 
12  ])  the  onboard  equipment  will  need  to  be  replaced  in 
order  to  accommodate  for  the  envisaged  changes  : 

•  Communications 

From  today’s  VHF  and  HF  voice  communication,  a 
transition  to  VHF  /  Mode  S  /  SatCom  datalink  with 
voice  backup  is  planned. 

•  Navigation 

The  navigation  presently  is  based  on  radio  navi¬ 
gation  (NDB;  VOR;  DME)  for  enroute  and  terminal 
operations  and  ILS  for  precision  approach  and 
landing.  This  will  change  completely  towards 
satellite-based  navigation  enroute  (GPS  / 
GLONASS)  and  MLS/GPS  precision  approach  and 
landing. 

•  Surveillance 

The  present  surveillance  system  of  primary  /  sec¬ 
ondary  radar  and  voice  position  reports  will  be 
replaced  by  a  mixture  of  ADS  (Automatic  De¬ 
pendent  Surveillance)  and  secondary  radar  (Modes 
A/C/S). 

•  Air  Traffic  Management  (ATM) 

The  current  air  traffic  system  with  its  fixed  route 
structure  and  separation  control  by  air  traffic  con¬ 


trollers  will  change  to  a  flexible  airspace  use,  where 
the  separation  responsibility  will  more  and  more  be 
transferred  to  the  aircraft  (the  so-called ,  J^ree 
Flight^  concept). 

The  ATM  system  operation  within  the  long-term  FANS 
concept  (for  details  please  refer  to  [  9  ])  will  involve  the 
type  of  airspace,  and  the  associated  airspace 
boundaries,  being  designated  dynamically  (i.e. 
designated  according  to  the  actual  traffic  demand  for 
the  relevant  period  of  time).  Direct  routing  will  be  used 
for  all  en-route  flying  (no  fixed  route  structure).  Two 
types  of  airspace  will  be  available: 

Autonomous  Airspace: 

•  only  used  in  low/medium  traffic  density  situations 

•  only  available  to  suitably  equipped  aircraft 

•  ground  ATM  system  ensures  that  traffic  density 
does  not  exceed  a  safe  level 

•  aircraft  follow  free  flight  trajectories 

•  conflict  detection  and  resolution  using  on-board 
system  (Super  Traffic  Alert  and  Collision  Avoid¬ 
ance  System  (Super  TCAS)) 

•  independent  safety  back-up  for  collision  avoidance 
provided  by  TCAS 

•  all  trajectories  and  modifications  downlinked  to  the 
ground  ATM  system  to  enable  overall  system 
monitoring 

Controlled  Airspace: 

•  will  apply  to  all  high  traffic  density  situations 

•  ground  ATM  system  ensures  that  traffic  density 
does  not  exceed  a  safe  level 

•  ground  ATM  system  responsible  for  conflict 
detection,  resolution,  system  monitoring 

•  conflict  resolution  instructions  uplinked  to  relevant 
aircraft  (resolution  responsibility  can  be  delegated 
to  aircraft  equipped  for  autonomous  operation) 

•  majority  of  aircraft  operating  to  4D  contracts 
negotiated  over  data  link 

•  less  well-equipped  aircraft  given  strategic  in¬ 
structions  over  data  link 

•  tactical  control  by  RTT  will  only  be  used  in  fail¬ 
ure/emergency  situations 

•  operational  strategy  ensures  that  best  equipped 
aircraft  receive  the  best  service  (i.e.  get  minimum 
disturbance  from  their  ideal  flight  path) 

This  future  ATM  situation  will  require  onboard 
equipment  with  the  following  capabilities: 

•  4D  trajectory  generation,  taking  into  account  pilot 
and  Airline  Operation  Control  (AOC)  requirements, 
airspace  restrictions,  any  ground  ATM  imposed 


23 


constraints,  and  also  any  conflict  resolution 
requirements  (for  autonomous  operation) 

•  4D  guidance  within  bubble  of  airspace  allocated  by 
ground  ATM  system,  or  along  ideal  trajectory  in 
Autonomous  airspace 

•  contract  negotiation  with  ground  ATM  system  over 
data  link 

•  performance  monitoring  relative  to  Required 
Navigation  Performance  (RNP)  for  current  airspace 

•  Precision  Position  Determination  -  using  GNSS 
(high-integrity  civil  system) 

•  Data  Link  using  Satellite,  VHP  and  Mode  S  links  to 
provide  two-way  link  between: 

•  Pilot/Controller  -  for  contract  negotiation, 
strategic  instructions,  meteo  forecasts 

•  Aircrafi/ATM  system  -  for  ADS  etc.  (no 
direct  controller/pilot  involvement) 

•  Aircraft/AOC  -  for  airline  operational, 
maintenance  etc,  purposes 

•  Aircraft/ Aircraft  -  for  autonomous  conflict 
detection  and  resolution 

•  Conflict  Detection  and  Resolution  -  a  Super  TCAS 
providing  medium-term  (5-10  minutes)  capability  in 
autonomous  airspace 

•  Collision  Avoidance  -  TCAS  to  provide  an  inde¬ 
pendent  short-term  safety  backup 

•  Cockpit  Human-Machine  Interface  -  displays  and 
inputting  capabilities  to  ensure  adequate  crew 
monitoring  and  control  of  FMS,  Data  Link,  etc.  in  a 
4D  negotiated  contract  environment,  and  suitable 
traffic  information  displays  for  operation  in 
autonomous  airspace 

While  some  of  the  CNS/ATM  feature  described  above 
are  to  be  seen  on  a  long-term  scale,  there  are  a  lot  of 
„first  steps“  in  the  near  future  that  force  airlines  (see  [  6 
])  and  other  commercial  aircraft  operators  to  invest 
heavily  in  new  equipment,  that  provides  : 

•  Reduced  Vertical  Separation  Minima  (RVSM), 

•  8.33  kHz  VHP  channel  separation, 

•  Traffic  Collision  Avoidance, 

•  VHP  /  Mode  S  /  SatCom  Datalink, 

•  Basic  Area  Navigation  (B-RNAV), 

•  MLS  /  GPS  approach  and  landing  . 

This  impressive  shopping  list  indicates  that  airline  and 
commercial  aircraft  operators  are  under  extreme 
investment  pressure  for  new  avionics  equipment  in 
order  to  be  prepared  for  the  new  air  traffic’ system. 


It  is  quite  obvious,  that  under  those  circumstances  the 
priority  of  electronic  crewmembers  to  support  the  pilots 
is  very  low. 

Or,  to  put  it  bluntly,  the  airlines  and  commercial  aircraft 
operators  presently  have  other  things  to  care  for  than  an 
electronic  crewmember  that  just  increases  flight  safety 
(..We're  safe  anvwav“)  and  decreases  pilot  workloM 
(..What  are  they  being  paid  for  ?*'). 

5.  CONCLUSION 

The  dilemma  is  quite  obvious.  On  one  hand  the  need 
for  intelligent  pilot  support,  implemented  through  an 
electronic  crewmember,  is  recognised  and  the  tech¬ 
nological  matureness  of  the  proposed  solutions,  such  as 
CASS Y,  has  been  demonstrated  in  flight.  On  the  other 
hand,  airlines  and  other  commercial  operators  consider 
EC’s  as  „nice-to-have“  for  the  time  being  . 
Consequently  they  are  not  prepared  to  invest  in  EC 
products  today.  And,  to  be  honest,  one  has  to  admit  that 
it  wouldn’t  be  a  small  investment.  Just  think  of  the  non¬ 
recurring  costs  for  a  software  with  more  than  200 
kLOCs  if  they  were  developed  according  to  the 
certification  standards  for  flight  essential  systems. 

The  way  ahead  might  therefore  be  : 

1 .  Introduce  the  basic  EC  functions  step-by-step  in 
commercial  avionics,  e.g  the  flight  management 
system,  without  increasing  the  prices.  This  might  be 
the  „winning  point“  for  the  customer  and  therefore 
pay  off. 

2.  Focus  on  military  transport  applications,  were  the 
technological  challenges  are  even  higher  and  the 
economical  pressure  on  the  operator  are  lower. 
Advance  thereby  the  state-of-the-art  and  try  to  have 
a  commercial  EC  as  spin-off. 

To  conclude  and  to  answer  the  question  of  the  title  :  the 
authors  of  this  paper  are  deeply  convinced  that  on  some 
day  in  the  future  fhe  human  crewmembers  will  be 
welcomed  on  their  flight  deck  by  an  electronic 
crewmember. 
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SUMMARY 

This  article  describes  a  generic  software  architecture  for 
the  interaction  between  the  modules  of  a  pilot  assistant 
system  based  on  shared  memory  blocks  and  a  central 
communication  control  module.  The  architecture  does  not 
depend  on  the  number  or  functionality  of  the  modules  and 
is  therefore  largely  independent  of  a  specific  application. 
The  consistency  of  shared  data  is  ensured  with  read  and 
write  locking.  Other  central  features  are  the  ability  to  easily 
integrate  additional  modules  at  runtime  and  the  special 
consideration  of  any-time-algorithms  to  support  planning 
modules  in  a  real  time  environment-  The  interaction  of  the 
modules  and  their  functionality  are  strictly  separated  to 
enhance  the  maintainability  and  to  simplify  the 
implementation  of  modules  as  well  as  to  improve  the 
reusability  of  the  architecture  for  different  assistant 
systems. 

1.  INTRODUCTION 

The  Institute  of  Flight  Guidance  of  the  German  Aerospace 
Research  Establishment  (DLR)  in  Braunschweig  is 
currently  developing  a  prototype  of  an  intelligent  pilot 
assistant  system  to  support  the  pilot  in  a  Free  Flight 
scenario  [1,  2].  This  system  features  new  functions  for 
autonomous  separation  assurance,  including  monitoring  of 
air  traffic  and  trajectory  planning. 

Currently  the  technical  possibilities,  as  well  as  the  potential 
benefits  and  dangers  of  Free  Flight  are  still  being 
investigated  and  a  clearly  defined  Free  Flight  scenario  does 
not  yet  exist.  The  exact  behaviour  of  the  assistant  system, 
however,  can  only  be  defined  once  the  Free  Flight  scenario 
has  been  laid  out  with  sufficient  detail.  Hence  the 
architecture  of  the  assistant  system  must  be  highly  flexible, 
so  that  the  impact  of  revised  design  decisions  will  be 
limited. 

Similar  to  current  experimental  pilot  assistant  systems  (e.g. 
CAMA,  GASSY  [3]),  the  basic  functions  of  the  Free 
Flight  assistant  system  will  be  a  human  machine  interface 
(HMI),  monitoring  (surrounding  traffic  and  on-board 
systems),  situation  assessment  (conflict  prediction,  system 
diagnosis)  and  trajectory  planning,  as  shown  in  figure  1.  In 
a  specific  assistant  system  the  function  block  templates  are 
substituted  by  one  or  more  function  modules.  The 
difference  between  a  Free  Flight  assistant  system  and 
previous  systems  is  not  necessarily  apparent  in  all  basic 
functions,  for  example  system  diagnosis  components  can 
still  be  used  in  the  Free  Flight  assistant  system,  since  the 
characteristics  of  the  aircraft  have  not  changed.  There  will, 
however,  still  be  some  work  required  for  integration  and 


adaptation  of  such  functions.  Since  the  Free  Flight 
concept  itself  is  still  a  research  issue  it  is  intended  to 
build  a  prototype  first,  which  will  have  functions  that 
probably  do  not  differ  from  the  final  system. 


Figure  1:  Basic  functions  of  a  pilot  assistant  system. 


For  these  reasons  we  are  developing  a  generic 
interaction  scheme  (software  architecture)  for  the 
assistant  system,  which  supports  the  direct  inclusion  of 
function  modules  from  other  systems.  Modules  which 
have  been  tested  in  the  prototype  can  then  be  integrated 
into  the  final  system  without  much  effort.  The 
complete  interaction  between  all  modules  is  the  main 
part  of  the  generic  design.  Fixing  this  interaction  by 
design  does  not  only  enable  reusing  modules  in 
different  assistant  systems,  but  it  also  simplifies  the 
maintenance  of  such  systems.  In  the  development  of 
new  modules  one  can  concentrate  on  the  functionality, 
since  the  entire  interaction  within  the  assistant  system 
has  been  developed  and  already  tested.  As  far  as 
message  flow  and  data  interchange  is  concerned,  there 
is  no  necessity  to  distinguish  between  different  kinds  of 
function  modules,  as  for  example  between  a  traffic 
monitor  and  a  free  flight  planner.  Therefore,  in  the 
following  we  will  speak  of  general  function  modules 
and  do  not  have  specific  modules  in  mind. 

Aside  from  the  requirements  mentioned  the  generic 
architecture  must  meet  some  more  criteria; 

•  The  architecture  must  support  the  inclusion  of  new 
function  modules  (e.g.  weather  monitor),  this 
should  be  possible  even  at  runtime. 

•  Modelling  system  behaviour  is  achieved  by 
modelling  module  interaction.  In  a  generic 
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architecture  the  interaction  must  not  explicitly  include 
specific  modules,  since  replacing  a  module  could 
otherwise  require  a  redesign  of  the  interaction. 

•  Function  modules  can  be  triggered  by  event  and/or 
periodically. 

•  The  use  of  planners  in  a  real  time  environment  places 
additional  requirements  on  the  interaction:  In  general 
there  will  be  time  limits  for  planning  tasks.  It  must  be 
also  possible  to  involve  changes  of  the  environment  in 
the  current  planning  process.  The  architecture  must 
allow  the  use  of  such  complex  modules  without 
constraining  their  functionality. 

In  this  paper  we  are  presenting  one  possible  variant  of  a 
generic,  object  oriented  architecture  for  module  interaction 
that  satisfies  these  requirements.  The  use  of  this 
architecture  is  by  no  means  limited  to  pilot  assistant 
systems,  it  could  be  used  for  any  system,  which  functions 
are  divided  into  several  modules. 

This  paper  will  not  treat  the  subject  of  object  oriented 
software  development.  For  the  important  topics  of 
inheritance  and  polymorphy  the  reader  is  referred  to  the 
literature,  e.g.  [4,5]. 

2.  MODULE  INTERACTION  AND  SITUATION 
REPRESENTATION 

From  an  implementational  point  of  view  it  is  desirable  to 
divide  the  system  into  separate  modules  (processes).  This 
can  significantly  ease  the  development  of  functions  and 
also  maintenance  of  the  system.  There  are  many 
possibilities  for  the  interaction  between  such  modules,  e.g. 
one  could  think  of  a  direct  communication  between  all 
modules.  Because  the  modules  run  concurrently  it  is  hard 
to  get  a  clear  picture  of  all  interaction  that  takes  place  over 
time,  and  therefore  the  integration  of  a  new  module  into  an 
existing  system  is  not  only  expensive  but  susceptible  to 
errors  and  inconsistencies.  Moreover,  using  direct 
communication  one  needs  to  know  about  all  participating 
modules  and  probably  has  to  make  changes  to  existing 
modules,  too. 

However,  a  module  must  not  know  about  the  existence  of 
other  modules  in  the  system  and  it  must  not  make 
assumptions  about  them.  This  would  limit  its  future  use  in 
another  system  and  also  a  runtime  reconfiguration  of  the 
assistant  system.  Hence  an  important  feature  of  a  module  is 
that  it  does  not  know  how  its  output  is  processed  by  other 
modules  and  also  where  its  input  is  coming  from.  In  order 
to  combine  a  number  of  modules  to  a  complete  system  a 
central  module  is  required,  which  we  call  Module  Manager 
(MM).  This  is  the  only  module  that  function  modules  can 
assume  to  exist  and  during  initialisation  each  function 
module  must  register  with  the  MM.  This  makes  all  active 
modules  known  to  the  MM.  All  data  that  shall  be 
transferred  between  modules  are  collected  in  the  central 
situation  representation,  which  will  be  called  Data  Pool 


(DP)  in  the  following.  In  order  to  be  able  to  follow 
events  in  the  system  without  making  assumptions  about 
other  modules,  they  can  bind  themselves  to  certain  data 
objects  within  the  DP.  In  this  case  the  module  will  be 
notified  whenever  the  object  changes  and  can  respond 
accordingly.  Instead  of  an  explicit  definition  of  the 
interaction  between  modules,  this  interaction  is 
implicitly  defined  via  data  objects: 

The  system  diagnosis  module  monitors  the  data 
from  the  aircraft  interface  and  in  case  of  a  failure 
generates  a  diagnosis  and  perhaps  corrected 
performance  data.  The  planner  reacts  to  changes 
of  the  performance  data  and  trajectories  of  other 
aircraft,  it  checks  whether  the  active  trajectory 
can  still  be  flown  and  suggests  an  alternate 
trajectory.  Finally  the  HMI  informs  the  pilot  of 
the  failure  and  that  an  alternate  trajectory  is 
suggested  by  the  system. 

In  this  simple  example  data  objects  are  Aircraft- 
Interface^Data,  Diagnostic-Result,  Performance-Data, 
Other-Trajectories,  and  Suggested- Alternate.  The 
modules  register  with  the  MM  informing  the  MM 
about  which  objects  they  will  read  (e.g.  planner  will 
read  Performance-Data  and  Other-Trajectories). 
Although  the  MM  does  not  know  about  the  functions 
of  the  modules,  it  knows  which  data  objects  are  read  by 
each  module.  The  MM  will  notify  all  registered  readers 
of  an  object  in  case  the  object  changes.  In  this  way  the 
planner  will  be  notified  of  the  changed  performance 
data  in  the  above  example.  Hence  the  use  of  a  module 
is  tied  to  its  input  and  output  data  which  are  defined  in 
the  module  implementation.  It  is  not  important  to  the 
planning  module  whether  the  performance  data  are 
generated  by  the  diagnosis  module  or  by  an  additional 
module  that  may  be  added  at  a  later  time. 


Figure  2:  Only  DP  and  MM  are  guaranteed  to  exist  in 
the  generic  architecture  for  module 
interaction. 
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Hence  the  core  of  the  assistant  system  consists  of  the 
Module  Manager  and  Data  Pool.  Any  function  module  can 
rely  on  their  existence  in  any  system  (Figure  2).  Aside 
from  providing  read  and  write  locking  mechanisms  the  DP 
maintains  a  list  of  readers  for  each  data  object.  In  the 
example  the  planner  and  the  HMI  are  readers  of  the 
Performance-Data,  while  the  diagnosis  module  is  its  writer. 
The  MM  uses  this  information  to  inform  registered 
modules  via  messages  about  changes  of  data  objects.  This 
functionality  has  been  realised  using  an  object  oriented 
programming  language  with  inheritance  and  encapsulation 
of  data.  Hence  access  violations  can  be  recognised  during 
compile  time  already.  Additional  control  mechanisms  are 
achieved  by  performing  consistency  checks  during  system 
initialisation.  Aside  from  the  event  triggered  notification 
described  above,  modules  can  also  be  triggered 
periodically. 

The  Module  Manager  offers  additional  benefits  for  the 
module  interaction.  All  messages  and  information  are 
passing  through  this  central  modul,  and  therefore 
mechanism  for  load  distribution  or  deactivation  of 
modules,  which  cannot  contribute  significantly  to  the 
solution  in  a  critical  situation,  can  be  easily  implemented. 
In  addition  the  system  behaviour  (e.g.  message  flow) 
becomes  more  transparent,  when  it  can  be  explained  by 
observing  just  one  module  rather  than  all  modules 
simultaneously.  This  also  enables  proving  a  specific  system 
behaviour  despite  of  using  concurrent  modules. 

Even  though  Module  Manager  and  Data  Pool  are  termed 
modules  in  this  paper,  they  may  not  be  implemented  by 
individual  processes.  It  is  also  possible  to  distribute  their 
functionality  to  the  individual  function  modules  of  the 
assistant  system. 


3.  MESSAGES 

In  the  previous  chapter  messages  have  been  mentioned 
already  in  connection  with  the  functionality  of  the  Data 
Pool.  In  case  a  data  object  changes,  the  DP  sends  messages 
to  all  modules  that  are  readers  of  this  object.  It  is,  however, 
not  always  desirable  to  send  such  messages  after  each  write 
operation,  since  then  other  modules  might  have  to  spend 
much  time  with  reactions  to  update-messages  even  though 
the  data  object  may  still  change  several  times  over  a  very 
short  period  of  time.  For  example,  a  planning  module  may 
be  planning  an  avoidance  trajectory  to  solve  a  conflict  that 
is  expected  in  eight  minutes  by  continuously  improving  an 
initial  trajectory.  The  other  modules  should  not  be 
informed  of  every  update  to  this  trajectory,  it  is  sufficient  if 
they  are  informed  once  the  planning  is  completed. 
Therefore  special  access  operations  are  provided  by  the  DP 
which  suppress  messages  to  other  modules.  Depending  on 
the  situation  a  module  can  then  use  the  appropriate  access 
operation. 


The  use  of  write  operations  without  messages  to  other 
modules,  however,  causes  another  problem. 
Considering  the  example  above  there  will  be  a  point  in 
time  at  which  the  planning  must  have  finished,  since 
otherwise  the  conflict  cannot  be  avoided  any  more.  In 
general  complex  planning  tasks  are  NP-complete,  i.e. 
the  time  required  to  compute  an  optimal  plan  depends 
exponentially  on  the  number  of  planning  parameters, 
e.g.  the  number  of  other  aircraft.  Therefore  it  cannot  be 
guaranteed  that  the  planner  has  completed  the  next 
improvement  to  the  plan  within  the  remaining  time  so 
that  it  may  not  be  able  to  issue  an  update-message  any 
more. 

This  problem  can  be  solved  by  adding  timers  to  update- 
messages.  Upon  expiration  of  the  timer  the  message 
will  be  sent  to  the  other  modules.  This  assures  that  the 
planning  result  at  the  moment  the  timer  expires  is 
available  to  the  other  modules,  even  when  the  planner 
itself  would  be  caught  in  an  endless  loop. 

Based  on  the  above  a  message  consists  of  the 
following: 

•  Sender  (e.g.  System  Diagnostics) 

•  Receiver  (e.g.  Module  Manager) 

•  Message  type  (e.g.  write  operation) 

•  Data  object  (e.g.  Performance-Data) 

•  Time  at  which  the  message  should  be  sent 

(Normally  messages  will  be  sent  immediately,  i.e.  the 
default  timer  value  is  "Now".) 

Among  others  our  system  architecture  uses  the 
following  message  types: 

•  Log_Read:  Register  as  reader  of  a  data  object, 

•  Unlog_Read:  Cancel  registration  as  reader, 

•  DO_Update:  data  object  has  been  updated, 

•  Continue:  continue  previously  interrupted  task  of  a 
module, 

•  Update_Timer:  update  timer  values  of  all  messages 
referring  to  the  same  data  object, 

•  Stop:  Terminate  process. 

4.  IMPLEMENTATION 

The  core  of  the  assistant  system  is  implemented 
without  any  knowledge  about  function  modules  to  be 
connected  later  on.  The  implementation  comprises 
realisation  of  the  MM  and  DP  as  well  as  a  template  for 
function  modules.  This  template  contains  placeholders 
for  the  actual  functions  of  the  modules,  while  the  part 
necessary  for  the  interaction  with  other  modules  is 
completely  implemented.  Hence  a  complete  module 
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that  is  immediately  usable  in  the  assistant  system  can  be 
built  rapidly  by  filling  the  placeholders  with  the  required 
functions.  Using  the  module  template  guarantees  that  the 
module  complies  with  the  communication  rules  and 
therefore  can  also  be  used  in  different  assistant  systems. 
The  prerequisites  for  this  approach  are  provided  by  using 
an  object  oriented  programming  language  (Use  of  abstract 
base  classes  and  polymorphism  [4]).  Hence  the  architecture 
is  generic  concerning  the  communication  between 
modules,  which  was  designed  for  arbitrary  assistant 
systems.  In  the  following  section  the  interaction  between 
modules  is  specified  in  more  detail. 


Function  Modules 

For  each  module  a  list  of  messages  is  managed.  The 
previous  chapters  then  imply  the  following  algorithm  for  a 
function  module,  which  is  shown  in  figure  3. 

Upon  initialisation  a  module  will  send  a  Log„read  message 
for  each  data  object  that  it  needs  to  read  and  then  waits  for 
incoming  messages.  These  are  processed  one  after  the 
other.  Each  module  will  have  a  function  "do_update''  to 
process  DO^Update  messages.  For  example  the  planner 
might  include  "do^update"  functions  for  the  performance 
data.  The  ‘’do_update"  function  is  meant  to  perform  only 
administrative  tasks  like  checking  which  action  must  take 
place.  The  actual  module  task  is  done  within  the 
"process_work''  function,  after  all  DO_Update  messages 
have  been  processed.  The  module  will  make  changes  to  the 
Data  Pool,  which  will  then  lead  to  DO__Update  messages  to 
other  modules,  only  from  within  the  "process_work” 
function.  Modules,  which  work  periodically,  also  include  a 
”send_delayed_continue"  function,  which  sends  a  Continue 
message  with  a  timer  value  to  the  Module  Manager.  The 
MM  will  then  send  a  Continue  message  back  to  the  module 
when  the  timer  expires. 


send_log_read() 

Repeat 

While  message  list  empty 

Wait  for  next  message 

For  all  messages  M  in  list 

Read  message  M 
Select  type 

DO_Update : 

Remove  all 

DO^Update  for  data  object 

do_update()  for 

data  object  of  message 

Continue: 

Remove  all  Continue 

messages 

process_work  () 

send_delayed_contmue() 
until  Stop  message 

send_unlog_read() _ 

Figure  3:  Algorithm  of  a  function  module. 


In  general  the  "do_update’'  functions  require  little 
computation  time.  This  is,  however,  not  true  for  the 
’’process_work"  function.  While  the  possibility  to  set 
timers  at  the  beginning  of  "process_work"  assures  that 
other  modules  receive  a  DO_Update  message  in  time, 
another  problem  arises  since  the  module  is  unable  to 
process  incoming  messages  itself  until  "process^work" 
returns.  For  example  the  result  of  a  planner  may  be 
completely  useless  after  a  single  change  in  the 
environment.  In  this  case  an  immediate  termination  of 
the  planning  process  would  be  appropriate.  On  the 
other  hand  a  change  in  the  environment  may  have  no 
influence  on  the  planning  result,  so  that  restarting  the 
planner  would  lead  to  the  same  result.  Only  the  planner 
itself  can  judge  whether  a  restart  is  appropriate  or  not, 
since  no  other  module  has  the  required  information 
about  the  internal  state  of  the  planner. 

Therefore  it  is  necessary  to  provide  the  possibility  to 
process  arriving  messages  within  the  "process_work" 
function.  This  can  be  achieved  by  replacing  the 
"process_work,  send  delayed  continue"  part  in  the 
above  algorithm  (figure  3)  with  the  algorithm  in  figure 


Repeat 

done  :=  process_work  () 

If  done 

send_delayed_continue  () 

Send  Message(S,MM,Update_Timer,-,Now) 
until  (done  or  message  list  not  empty) 

4. 

Figure  4:  Modification  of  algorithm  to  handle  new 
messages  while  process  is  working. 

With  this  modification  it  can  be  controlled  by  the 
implementation  of  the  "process_work"  function,  at 
which  points  the  module  can  process  incoming 
messages.  If  the  "process_work"  function  of  a  module 
requires  only  little  computation  time,  it  will  always 
return  "true"  (ready).  In  other  cases  the  function  will 
perform  only  some  part  of  its  calculations  and  then 
return  "false"  (not  ready).  The  module  can  then  process 
incoming  messages  and  "process_work"  can  resume. 
All  DO__Update  messages  will  be  set  to  a  timer  value 
of  "Now"  by  the  Update_Timer  message  once 
"process_work"  finishes. 

All  modules  of  the  assistant  system  follow  the  same 
interaction  algorithm.  A  complete  module  can  be  built 
by  implementing  the  “send__log_read“, 
“send_delayed_con-tinue“,  “do_update“  and 
“process_work“  functions. 


Data  Pool 

The  DP  is  an  active  part  of  the  interaction.  It  provides 
write  locks,  which  send  messages  to  the  MM  when 
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Repeat 

While  message  list  is  empty  and  time  of  first 
element  of  TL  is  not 

reached 

Wait  for  next  message  or  time 

For  all  M  in  TL  with  time  "Now" 

Insert  M  into  list  of  messages 
Remove  M  from  TL 

Get  M  from  list  of  messages 


If 

Update_Timer) 


(Time  later  than  "Now")  and 
(Type  not  equal 


Insert  M  into  TL 

else 

select  Type 

case  Update_Timer: 

For  all  messages  for  same 


object 


Remove  from  TL  or  list  of  messages 

Replace  time 

case  Log_Read: 

Insert  sender  in 

list  of  readers 

case  DO_Update: 

For  all  readers  R 

of  data  object 

Send 

DO_Update  message  to  R 

case  Continue: 

Send  Continue  Message  to 

sender 

case  Unlog_Read: 

Remove  sender 

from  list  of  readers 
until  Stop  message 


requested  by  the  function  modules.  Write  locks  can  send 
messages  immediately  after  the  object  has  been  modified, 
messages  can  be  delayed  (message  with  timer),  or  no 
message  is  sent  at  all.  It  is  up  to  each  module  to  select  the 
appropriate  write  lock  based  on  the  relevance  of  the  object 
modification  in  the  current  situation  and  context. 


Module  Manager 

In  addition  to  the  list  of  messages  to  be  processed  the 
Module  Manager  also  maintains  the  timer>message  list  TL, 
which  is  sorted  by  timer  value.  The  algorithm,  shown  in 
figure  5,  applies. 


The  Module  Manager  waits  for  new  messages  or  is 
activated,  when  one  of  the  timers  expires.  In  this  case  all 
messages,  whose  timer  has  also  expired  are  inserted  into 
the  list  of  messages.  Then  all  messages  are  processed  in 
order.  If  a  message  lies  in  the  future,  this  message  includes 
a  timer  and  is  therefore  inserted  into  the  timer  list  TL.  The 
only  exception  is  the  Update^Timer  message.  It  is  always 


processed  immediately.  The  timer  value  indicates  how 
the  corresponding  timer-messages  should  be  modified. 
(In  general  the  timer  value  will  be  set  to  "Now",  since 
the  "process_work"  function  of  the  sender  is  done.) 

To  trigger  modules  periodically,  timers  can  be  used  to 
send  delayed  Continue  messages.  The  MM  returns 
Continue  messages  directly  to  the  sender.  Therefore,  in 
the  function  module  function  „send_delayed_continue“ 
a  module  can  use  delayed  Continue  messages  to  trigger 
itself  after  the  timer  has  expired. 


Figure  5:  Module  Manager  algorithm. 


5.  SUMMARY  AND  OUTLOOK 

Growing  demands  on  the  aircrew  due  to  increasing 
traffic  volume  or  changes  in  ATM  structures  lead  to  the 
requirement  for  pilot  assistant  systems  with  new 
functionality.  This  will,  however,  make  not  all 
components  of  current  systems  obsolete  (monitoring, 
situation  assessment,  planning,  HMI),  so  that  it  is 
advisable  to  borrow  functions  for  new  systems  form 
current  systems. 

In  this  paper  a  software  architecture  for  a  generic 
interaction  scheme  has  been  presented,  which  provides 
a  clear  separation  between  module  interaction  and 
module  functionality  in  a  pilot  assistant  system.  In 
addition  to  simplify  testing  and  maintenance  of 
individual  modules  this  separation  also  reduces  the 
development  effort  for  new  assistant  systems.  The 
implementation  of  the  module  interaction  serves  as  a 
common  basis  for  various  assistant  systems,  which  on 
the  one  hand  allows  to  integrate  existing  modules  into 
new  systems  very  easily,  and  on  the  other  hand  reduces 
the  effort  for  the  development  of  new  modules  by 
concentrating  on  their  functionality.  The  dependencies 
between  the  modules  are  modelled  via  their  shared  data 
objects,  modules  declare  themselves  as  reader  or  writer 
of  these  objects  during  initialisation.  If  a  data  object  is 
used  by  several  modules,  the  reader  modules  are 
implicitly  dependent  on  the  writer  module.  The 
mechanisms  for  interaction  are  designed  in  such  a  way 
that  special  requirements  of  real  time  systems  (e.g. 
any-time-planner)  are  taken  into  account.  Future 
enhancements  to  the  interaction  are  also  simplified, 
since  they  are  implemented  at  one  central  location 
rather  than  in  each  individual  module.  This  is  a 
prerequisite  for  the  desired  longevity  of  the  concept. 

The  core  system  is  currently  being  developed  and  will 
be  implemented  in  a  pilot  assistant  prototype.  Building 
on  this  prototype  the  flexibility  of  the  concept  will  be 
utilised  to  build  a  final  assistant  system  for  a  Free 
Flight  scenario.  At  that  point  in  time  the  core  system 
will  have  accumulated  a  number  of  testing  hours  in  the 
prototype,  so  that  a  reduced  effort  for  the  development 
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of  the  interaction  is  expected,  even  though  the  system  is 
growing  in  complexity. 
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Overview 

•  What  is  Hazard  Monitor? 

•  Why  is  Hazard  Monitor  needed? 

•  Brief  Hazard  Monitor  history 

•  Which  types  of  problems  does  HM  help  resolve? 

•  Which  types  of  problems  does  HM  not  help  resolve? 

•  How  Hazard  Monitor  Works 

•  Expectation  Network  Structure 

•  Sample  Expectation  Networks 

•  Hazard  Monitor  Architecture 

•  Hazard  Monitor  System  Tools 

•  HM  Technology  Differentiators 

•  Possible  Applications 

•  Summary 

•  List  of  Acronyms 


What  is  Hazard  Monitor? 

Hazard  Monitor  (HM)  is  a  knowledge-based  system  that  alerts  human  operators  to  discrepancies  between 
actual  and  expected  system  states  in  a  human-centered  and  context-sensitive  manner  before  the  adverse 
consequences  of  the  discrepancies  become  unavoidable. 

HM  uses  existing  alerting  mechanisms  to  notify  pilots  of  problems  in  time  to  take  corrective  actions,  if 
the  pilots  choose  to  do  so.  It  is  a  "discrepancy  highlighter." 

Think  of  HM  as  a  situation  awareness  enhancer. 

Human-centered  and  context-sensitive  means  -  take  all  available  data  and  examine  them  from  the 
human’s  goal-oriented  and  hazard-avoidance  perspective.  Therefore,  the  severity  of,  and  proximity  to,  the 
adverse  consequences  are  factors  in  determining  the  alerts  and  their  level  (warning,  caution,  advisory). 

Why  is  Hazard  Monitor  Needed? 

•  Complex  systems  and  environments,  opaque  interfaces  and  over-automation  make  errors  likely. 
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•  Accidents  do  not  typically  occur  due  to  single,  isolated  errors.  They  occur  due  to  a  "chain  of 
events"  that  the  human  operator  does  not  detect  or  does  not  fully  appreciate  (level  1:  detection, 
level  2:  comprehension  and  level  3:  projection  -  SA  problems). 

•  Traditional  error  strategies  (structured  design,  ergonomics,  personnel  selection,  traditional  training 
and  automation)  have  reached  limits  of  effectiveness. 

•  Accidents  are  not  the  only  problem.  System  inefficiencies  result  from  these  problems,  much  more 
often  than  do  accidents. 


•  Blaming  pilots  is  not  the  answer!  Helping  pilots  avoid  consequences  is  part  of  the  answer. 
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There  is  a  questionable  advantage 
to  the  introduction  of  more 
technology  as  "intermediary".  This 
only  distances  the  operator  from 
the  system  which  is  to  be 
controlled  and  monitored. 

Instead,  HM  is  designed  as  an 
associate  system  pilot  aid, 
providing  an  "extra  set  of  eyes"  to 
the  crew.  Its  purpose  is  to  support 
pilot  roles. 


Brief  Hazard  Monitor  History 

•  Pilot’s  Associate  (PA)  -  DARPA/USAF  1985-1992  (USAF  contract  F33615-85-C-3804) 

•  AI  used  to  enhance  fighter  pilot  SA  and  improve  mission  effectiveness. 

•  HM  was  called  "Error  Monitor".  It  was  one  small  part  of  the  Pilot  Vehicle  Interface  (PVI), 
which  was  1/5*  of  PA. 

•  NASA  SBIR  -  NASA-Langley  1992-1995  (NASA  Contract  NAS  1-20210) 

•  Error  Monitor  used  to  help  airline  pilots  deal  with  complex  FMS  interactions. 

•  Integrated  with  Boeing  747-400  FMS  made  by  Honeywell. 

POS INTT  errors,  waypoint  errors,  descent  and  approach  hazards. 

•  The  name  Error  Monitor  (EM)  was  changed  to  Hazard  Monitor  (HM)  to  remove  the  notion 
of  blame. 

•  USAF  SBIR  -  Wright  Lab  1994-1997  (USAF  contract  F33615-95-C-361 1) 

•  A  new  arbitration  and  presentation  managment  capability  was  added  to  handle  subtle 
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notification  inter-dependencies;  duplicates,  priorities,  conflicts,  abstraction. 

•  Networks  implemented  included;  emergency  and  normal  descent,  safe  airspeed,  safe  cabin 
altitude,  altimeter  setting,  normal  approach,  and  go-around  hazard  networks  for  military 
transport. 

•  USAF  SBIR  -  Armstrong  Lab  1997  (USAF  contract  F41624-97-C-5027) 

•  The  HM  design  was  enhanced  to  support  intelligent  tutoring;  real-time  aiding,  and  post¬ 
flight  debriefing. 

Which  types  of  problems  does  HM  help  resolve? 

•  There  are  common  problem  characteristics; 

•  Evolving  situation  with  subtle  cues 

•  Time  for  human  to  recognize  and  act  to  avoid  bad  consequences 

•  Advantage  to  consistency  checking 

•  Utility  in  detection  enhancement 

•  FAR,  SOP  compliance  especially  during  goal-oriented  activity  For  example,  "approach 
gates"  compliance 

•  HM  provides  a  general  solution  to  a  range  of  problems,  not  a  point  solution  to  specific  problems. 


•  Pilot  technique  accomodated 

•  We  explicitly  acknowledge  the  need  for  pilot  authority  &  creativity 

•  Allow  the  operator  to  choose  and  implement  a  strategy 

•  Pilot  knows  what  to  do  when  a  potential  problem  is  identified 

•  Assumes  sufficient  data  available  electronically 

•  Airplane  data  bus,  ATC  data-link,  NAV  and  terrain  data  bases 

Which  types  of  problems  does  HM  not  help  resolve? 

•  Specific  problems  already  being  addressed  by  very  capable  systems 

•  CHT  (GPWS),  mid-air  collisions  (TCAS) 

•  Catastrophic  events 

•  Subsystem  failures;  e.g.,  fan  blade  failure 

•  Weather;  e.g.,  lightning  strikes 

•  Communication  problems 

•  ATC,  CRM 

•  Subsystem  diagnoses 

•  We  do  not  want  to  replicate  built-in  test  equipment 

•  Maintenance,  repair  or  overhaul  (MRO)  discrepancies 
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How  Hazard  Monitor  Works 

Pilot-Centered  monitoring  &  presentation  is  made  possible  through  Persistence  and  Context-Sensitivity 
•  Persistence 

•  The  pilots  are  notified  of  discrepant  states  leading  to  potential  hazards  upon  detection  and 
for  as  long  as  monitoring  is  performed. 

•  As  the  context  changes  (increasing  proximity  or  severity  of  the  consequences),  HM  re¬ 
sends  the  alert,  with  an  increased  severity  level  if  appropriate. 

•  Context  -  Sensitivity  is  defined  by  the  elements  which  comprise  the  networks: 

•  Expectation  Networks  describe  goal-directed  activities  and  procedures  which  are 
monitored. 

•  Initiators  describe  the  context  (conditions)  for  beginning  to  monitor  an  activity  (example: 
inside  initial  approach  fix). 

•  Terminators  describe  the  context  for  ending  monitoring  due  to:  goal  achievement  (ex. 
landed)  or  abandonment  (ex.  go-around). 

•  Situation  Nodes  describe  the  sub-contexts  within  activities  that  are  meaningful  to  the  pilot: 
locations  where  new  states  should  be  monitored;  irrelevant  states  no  longer  monitored; 
convenient  location  to  adjust  severity  of  notification  if  appropriate,  situation  nodes 
represent  the  current  context  up  to  the  point  where  right-hand  node  becomes  new  current 
context.  ( there  is  implicit  coverage  of  the  context  /  state  space  between  nodes) 

•  Expectations  describe  the  aircraft  states  and  their  expected  ranges  within  the  context  of  the 
activity.  Failed  expectations  trigger  canidate  notifications  for  presentation  to  the  pilots. 


Expectation  Network  Structure 

The  Harzard  networks  (also  called  expectation  networks)  provide  the  scaffolding  for  the  efficient 
encoding  of  domain  expertise.  Each  network  represents  a  normative  model  for  an  associated  activity  or 
procedure. 

The  network  does  not  implement  an  electronic  checklist.  HM  and  the  domain  knowledge  are  engineered 
so  that  the  system  remains  silent  the  unless  pilot  is  late.  This  accommodates  pilot  technique. 

The  structure  supports  incremental  knowledge  refinement  and  the  addition  of  new  knowledge. 

All  sub-contexts  within  an  activity  or  procedure  need  not  be  modeled,  only  those  where  a  domain  expert 
indicates  significant  discrepancies  should  be  brought  to  the  attention  of  the  crew. 
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Sample  Expectation  Networks 


While  we've  concentrated  on  hazard  prone  descent  and  approach  phases  of  flight,  HM  can  be  configured 
with  networks  relevant  to  other  flight  phases.  In  addition,  HM  can  accomplish  monitoring  for  safe 
configuration  and  safe  limits  during  all  phases  of  flight. 


1.  Normal  descent  network 

•  Runway  in  route 

•  MCP  altitude  set 

•  Wl/descent  rate 

•  Configuration 

•  Thrust  (with/without  icing) 

2.  Safe  airspeed  network 

•  Gear  over-speed 

•  Haps  over-speed 

•  Airspeed  exceeds  Vmo 

•  Airspeed  >  250  below  10,000’ 

•  Airspeed  less  than  Vmin 

•  Airspeed  fault  (pitot  tube  icing) 

3.  Safe  cabin  altitude  network 

•  Cabin  altitude  exceeds  10,000 

•  Cabin  altitude  exceeds  14,000 

•  Cabin  altitude  exceeds  25,000 

4.  Emergency  descent  network 

•  VVI/descent  rate 

•  Configuration 


5.  Descent  altimeter  setting  network 

•  Altimeter  still  set  at  29.92 

•  AGL  and  MSL  disagreement 

6.  Normal  approach  network 

•  Approach,  ranway  entered 

•  ILS  frequency  entered 

•  Haps,  gear  set 

•  Airspeed 

•  Inbound  course 

•  WI 

•  If  multiple  discrepancies, 
"Unstable  Approach" 

7.  Go  around  network 

•  Go  around  thrust 

•  Go  around  airspeed 

•  Go  around  VVI 

•  Go  around  pitch 

•  If  multiple  discrepancies, 

"Sink  Warning" 
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Hazard  Monitor  Architecture; 
Input  /  Processing  /  Output 


Software 

Iider&ce 

Knowledge 

Parametric 

Data 

Base 

E:9ectatioiL 

Network 

Knowledg^ 

Conflict/ 

Abstraction 

Raw  NcwState  p^ise 

_ lutei&ce  and  - 

AviDiocs  Parser 


State 

State 

*  Assessor 

Data  Model 

Presentation 

Ej^cctation  Candidate  ^  Ma,agement 

Netarork  “d  AiMtrali»n  x„acAWS 

Monitor  NotUKanon*  _ _ | 


*  Parse  Avionics  to  form 
partial  State  Vector  (SV)  with 
bw-hvelp  arsed  states. 

*  Isolate  core  of  HM  from 
specifics  of  o  utside  wo  rid : 

IEEE  488 

SCRAMNet 

TCP/IP 

AEINC/Ma4553 


*  Assess  M^bi^dassessinents 
from  logical  and  arithmetic 
fusbnof  lowbvristates. 

*  Location  to  perform  complex 
assessments,  trending,  and 
prediction. 

Standardizes  SV  across 
different  AC  types. 

*  Coii^fctes  Sttate Vector 

*  TT«iiiliw»  on  schfriiiW  hash 


*Eipectaiionni!ts  encode 
noimatwe  model  of  activty, 
procedure,  ox  safe  limits. 

*  A riiial avionics  compared 
to  cgqiected  states. 

*  DhcKcp  ancjBS  result  in  candidate 
set  of  notifications. 

^Monitor  supports  efficient  and 
p  aiallelpiocessing  of  networks. 


Presentation Manager  le^  onsb  le 
for  dispatchand  retraction  of  msgs 
on  AGAWS /EICASdisplay. 
‘^Adbitrationhandles  duplicate 

removal,  prioritization  (Iw  eiO,  c  onfKct 
sippression,  and  abstraction  to meta- 
nottficatiDn. 

*  Logcdetafled  disci^ancy  information 
whennoti&atbns  dispatched  to  pilots. 


Hazard  Monitor  Architecture: 

Class  /  Object  Diagram 

Domain  knowledge  is 
"instantiated"  in  memory 
during  system  initialization. 

Input,  Process,  Output  and 
Support  phases  of  processing 
proceed  in  a  round-robin 
fashion. 

Interface  objects  call  upon 
intermediate  Router  and 
Driver  objects  to 
communicate  with  external 
processes. 

The  State  Data  object 
represents  the  state  vector 
accessible  by  the  rest  of  the 
architecture. 

Presentation  Manager  and 
Arbitration  objects  log 
notifications,  and  reduces  the 
set  of  alerts  presented  to  the 
output  device  (ACAWS, 
EICAS). 
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Hazard  Monitor  System  Tools 

Validator  and  Visualizer 


Parses  Knowledge  Base  files 
comprising  expectation  networks. 
Performs  syntax-checking  and  error 
reporting.  Displays  a  graphical 
representation  of  the  hazard  networks. 
A  mechanism  to  examine, 
communicate,  and  produce  hard-copy 
documentation  of  the  configuration  of 
the  domain  expertise. 


Prober 


An  extension  of  the  Visualizer  tool. 
Reveals  real-time  processing 
behaviors  of  Hazard  Monitor  for 
inspection  during  configuration 
testing.  Updates  graphical 
representation  of  nets  via  color 
highlighting  and  flood  fills  based  on 
truth- value  of  network  elements. 
Provides  mechanism  to  selectively 
examine  state  vector  contents  in  real 
time. 
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Debriefer 


Visualizes  the  contents  of  the  detailed 
notification  log  created  by 
Presentation  Manager  object.  Log  file 
written  as  HM  monitors  for 
discrepancies  leading  to  hazardous 
consequences  and  updates  the  display 
device.  Log  includes  as  a  minimum, 
the  following  data;  time  message 
dispatched  to  display,  time  retracted, 
reason  retracted,  alert  severity  level, 
associated  context  (network,  situation, 
expectation).  List  format  supports 
sorting  on  column 

(ascending/descending)  -  notification 
log  easily  imported  into  commercial 
spreadsheet  packages.  Graphical 
format  depicts  log  information  about 
time-line  or  potentially  other  relevant 
taxonomies  (aviate,  navigate, 
communicate,  manage  systems,  safety, 
SOP,  performance,  economy,  network, 
situation). 


Simulation  with  CAWS  display 


Pictured  here  is  a  modified  version  of 
a  desk-top  simulator  program  which 
provides  a  means  to  test  HM 
configurations  against  "canned" 
scenarios  or  in  an  unscripted  manner 
under  the  Windows  NT  operating 
system. 

A  generic  CAWS  display  is  used  to 
visualize  the  output  of  HM.  It  provides 
priority  sorted  list  support  for  warning, 
caution,  and  advisory  notification.  A 
message  "explain"  capability  is  also 
available  by  double  clicking  on  the 
message  of  interest. 


HM  Technology  Differentiators 

•  Pilot-centered,  context-sensitive  monitoring:  efficiently  accommodates  pilot  technique  and 
authority.  Silent  unless  certain  that  pilot  late. 

•  Domain-level  configuration:  level  of  specificity  /  abstraction  appropriate  for  customer 
programmability  requirements.  The  objects  manipulated  are  mnemonically  familiar  to  domain 
experts. 


40 


•  Object-oriented  design;  networks  encapsulate  functionality  and  support  addition  of  new  attributes; 
nets  support  incremental  KB  development  and  refinement;  elements  of  architecture  can  be 
replaced  with  minimal  impact  to  the  rest  of  the  system. 

•  KBS  separation  of  code  and  data:  data  is  configured  by  end-user;  code  implementing  functionality 
developed  and  certified  once;  end-user  would  pick  and  choose  from  a  collection  of  certified 
elements;  parametric  data  store  provides  a  means  to  easily  adjust  limits,  dead-hands,  and  other 
comparison  values. 

•  Resource-friendly  processing:  monitoring  is  conducted  only  in  the  context  in  which  associated 
activities  are  being  pursued;  networks  could  support  parallel  processing;  complex  trending 
assessments  scheduled  at  configured  rates. 

•  Tool  set  supports  knowledge  life-cycle:  Validator  /  Visualizer  for  configuration.  Prober  for 
dynamic  processing  behaviors.  Debriefer  for  post-flight  discrepancy  analysis. 

•  Capable  of  multi-sensor  data  fusion;  not  single  sensor  /  single  alert  technology;  framework  for  de- 
confliction  of  notifications  and  abstraction  to  meta-level  notifications. 

•  Rexible  logging  and  recording  of  notification  events  linked  to  state  vector  snapshots.  Used  by 
debriefing  tools  for  reflection  on  performance  to  uncover  gulfs  in  proficiency  /  technique,  poor 
SOP,  or  loss  of  SA. 

•  Multi-use:  military  and  commercial  real-time  piloting  aid  apphcations,  intelligent  tutoring,  post¬ 
flight  FOQA  analysis. 

Possible  Applications 

General  Pilot  Aiding 

•  Initial  focus  would  be  on  non-essential  monitoring  and  alerting  in  a  focused  application;  attitude, 
AS,  trim  run-away,  flaps,  gear,  thrust,  fuel. 

•  Plan  would  be  to  validate  HM,  certify  it,  and  install  it. 

•  Fly  pilots  with  and  without  HM  in  high-fidelity  simulator  environment  to  measure  performance 
differences. 

•  Questions  to  be  answered:  Does  HM  really  help  enhance  SA?  Do  pilots  recognize  problems  more 
quickly?  Do  they  recover  more  accurately?  Might  HM  aggravate  complacency  problems? 

•  Plan  would  call  for  testing  with  next  generation  commercial  or  military  Flight  Management 
Systems. 

Partitioned  Monitoring 

•  Customer  specific  configurable  partition  for  tailoring  SOP  or  compliance  gate  monitoring. 

Specialized  Monitoring 

•  High-level  assessment,  monitoring  and  alerting  based  on  the  integration  of  muti-sensor  data. 

•  Predictive  capability  -  "don’t  tell  me  the  flaps  have  been  over-sped,  tell  me  if  and  when  an  over¬ 
speed  condition  is  likely  to  occur.  Safe  until  late  is  OK,  but  if  you  can  provide  time-horizon 
prediction  -  do  so." 

Pilot  Training 

•  Implement  a  version  of  the  Intelligent  Tutoring  System  prototype  which  can  be  used  to  check 
pilots  against  a  representative  hazard  set,  or  infrequently  encountered  checklists. 

•  Aiding  component  could  drive  multi-media  training  /  tutorial  system. 

•  Reduce  student  and  instructor  pilot  time;  maximize  effectiveness  of  simulator  time;  provide  for 
rich  post-flight  review  of  performance. 

Post-flight  analyses 

•  Data  reduction  -  HM  used  to  reduce  QAR  data  sets  from  commercial  fleet  for  detailed  analysis. 
Flexible  exceedences  specification. 

•  Trending  (FOQA)  -  Implement  customer  requirements  to  collect  and  trend  QA  information. 
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Automatic  Dependent  Surveillance-Broadcast  (ADS-B) 

•  FANS  /  Conflict  Probe  -  Route  conformity  /  altitude  discipline,  ATC  up-link  consistency 
checking. 

•  Violations  and  restrictions  checks  would  need  prediction  capability. 

Information  Management 

•  Display  highlighting  and  reconfiguration. 

•  Flexible  selection  of  notification  modality  -  monaural  audio,  text,  speech,  3-D  stereo,  synoptic 
displays,  animated  displays. 

Adaptive  Aiding 

•  Authorized  system  state  changes  /  commands  by  consent. 

Summary 

Unique  Hazard  Monitor  features: 

•  Pilot-centered,  context-sensitive  monitoring 

•  Procedure  de-confliction 

•  Sensor  disagreement  aiding 

•  Focus  on  hazard-prone  descent  and  approach  phases 

•  Preparatory  guidance 

•  Debriefing  support 

•  Performance  evaluation 
Aiding  prototype  ready  for  evaluation! 

•  Easily  integrated  into  simulations  and  actual  avionics 

•  Portable  demonstration  and  testing  environment 
Training  prototype  ready  for  implementation! 

•  Draw  upon  pre-existing  functionality,  commercial  software,  and  knowledge  base  development 
tools  and  processes 


List  of  Acronyms 

ACAWS  Advisory,  Caution,  and  Warning  System 

MRO 

Maintenance  Repair  or 
Overhead 

ADS-B 

Automatic  Dependent  Surveillance  Broadcast 

NASA 

National  Aeronautics  and 
Space  Administration 

ATC 

Air  Traffic  Control 

NAV 

Navigation 

CAWS 

Caution,  Advisory  and  Warning  System 

PA 

Pilots  Associate 

CFIT 

Controlled  Flight  Into  Terrain 

POS 

Position 

CRM 

Crew  Resource  Management 

PVI 

Pilot  Vehicle  Interface 

DARPA 

Defense  Advanced  Research  Projects  Agency 

QA 

Quality  Assurance 

EICAS 

Engine-Indicating  and  Crew- Alerting  System 

QAR 

Quick  Access  Recorder 

FANS 

Future  Air  Navigation  Systems 

SA 

Situation  Awareness 

FMS 

Flight  Management  Systems 

SBIR 

Small  Business  Innovation 
Research</TD 
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FOQA 

Flight  Operations  Quality  Assurance 

SOP 

Standard  Operating  Procedure 

GPWS 

Ground  Proximity  Warning  System 

TCAS 

Traffic-alert  and  Collision- 
avoidance  System 

HM 

Hazard  Monitor 

USAF 

Unitied  States  Air  Force 
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Initialize 
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KB 
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MCP 
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Design  for  a  Knowledge-Based  Decision  Support  System  to  Assist  Near 
Littoral  Airborne  Early  Warning  (AEW)  Operations 


Roberto  Zanconato 
Cambridge  Consultants  Ltd 
Science  Park 
Milton  Road 
Cambridge  CB4  4DW 

Abstract: 

This  paper  describes  the  design  and  early 
development  of  a  demonstrator  Knowledge- 
based  Decision  Support  System  (DSS)  to 
support  near  littoral  helicopter  based 
Airborne  Early  Warning  (AEW)  operations. 
The  scope  of  the  application  is  large, 
encompassing:  Detection,  Situation 

Assessment,  Reporting,  Tactical  decisions. 
Fighter  control.  Radar/sensor  handling  and 
Airmanship. 

The  design  has  focused  on  the  requirement 
to  provide  extensive  modularity  to  support 
extensibility  and  component  reuse  (both  of 
the  underlying  expert  knowledge  and  the 
associated  implemented  modules).  The 
modules  have  been  identified  as  the  core 
functionality  required  to  perform  the  tasks 
involved  in  an  integrated  manner.  A  multi¬ 
agent  approach  has  been  chosen  which 
follows  the  guidelines  laid  out  in  the 
Common  KADS  agent  and  communication 
models,  and  which  uses  as  its 
implementation  language  the  D-Muse 
distributed  real-time  AI  toolkit.  D-Muse 
provides:  object,  blackboard,  rule-based  and 
distributed  programming,  to  support 
modular  development;  and  run-time  process 
management  providing  capability  for 
dynamic  reconfiguration,  which  supports  the 
efficient  use  of  resources  and  error  recovery. 

The  paper  describes  how  the  many 
functional  components  of  the  proposed  DSS 
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are  supported  within  the  proposed  agent 
framework,  and  how  the  implementation 
will  be  realised  using  the  features  of  the  D- 
Muse  toolkit.  The  paper  also  touches  briefly 
on  how  the  complexity  of  the  task  and  the 
work  involved  was  estimated  from  the 
knowledge  gained  from  close  liaison  with 
AEW  operators. 


Keywords: 

AEW,  Real  time.  Knowledge  based. 
Decision  Support  System,  agent,  knowledge 
acquisition,  REKAP,  D-Muse,  PCPACK, 
KQML,  KADS. 


1  OVERVIEW  AND 

OBJECTIVES 

1.1  Overview  of  the  AEW  Task 

Figure  1  shows  a  snapshot  of  a  typical  AEW 
scenario,  in  which  AEW  helicopters  are 
responsible  for  protecting  an  advancing 
naval  task  force. 

The  primary  elements  of  the  scenario 
include: 

a)  the  task  force  being  protected; 

b)  a  frigate  forward  of  the  main  force 
hosting  the  AEW  aircraft; 

c)  a  further  frigate  well  advanced  of  the 
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Figure  1:  Typical  Naval  AEW  Scenario 


task  force  used  as  a  weapons  platform 
against  incoming  threats; 

d)  an  AEW  helicopter  ahead  of  the  task 
force  searching  for  contacts  and 
controlling  the  AEW  operation; 

e)  fixed  wing  intercept  aircraft  well  ahead 
of  the  force,  in  combat  air  patrols 
(CAPs)  defined  by  the  AEW  helicopter, 
awaiting  intercept  request  from  the  AEW 
helicopter; 

f)  further  frigates  on  the  task  force  flank 
used  for  ASW  towed  array  operations 
(passive  sonar);  and 

g)  further  helicopters  ahead  of  the  task 
force  performing  active  sonar  dipping  in 
ASW  operations. 

It  is  the  task  of  the  AEW  helicopter  to 
classify  as  early  as  possible  all  hostile 
airborne  contacts  in  the  vicinity  of  the  task 
force,  and  to  initiate  the  prosecution  of  those 
contacts  it  believes  represents  a  threat  to  the 
task  force,  by  efficient  management  of 
available  assets. 


1.2  Objectives  of  the  Demonstrator 

The  purpose  of  this  Decision  Support 
System  (DSS)  is  to  develop  a  system  which 
can  be  used  to  assess  the  expected  increase 
in  operational  effectiveness  of  the  Future 
Organic  Airborne  Early  Warning  (FOAEW) 
platform  when  operators  are  supported  in 
their  task  by  intelligent  decision  aids.  From 
a  successful  demonstration  of  capability  and 
improved  effectiveness  of  operators,  is  the 
potential  of  moving  the  application  toward  a 
full  mission  system. 

It  is  important  to  note  that  the  proposed 
decision  aid  is  not  intended  as  an 
autonomous  system  with  which  the  FOAEW 
operator  has  minimal  interaction.  Instead,  it 
is  required  to  be  a  co-operative  system  in 
which  the  system  and  operator  are  able  to 
utilise  the  skills  most  appropriate  to  their 
capabilities.  Even  if  it  were  technically 
feasible  to  provide  an  autonomous  system,  it 
is  not  obvious  that  this  would  be  a  desirable 
feature,  since  such  a  system  would  risk 
leaving  AEW  operators  with  no  obvious 
control  or  responsibility  over  the  progress  of 
the  sortie. 

2  FOAEW  DSS  SCOPING 

The  bases  of  the  application  scoping  was  the 
results  obtained  from  many  Knowledge 
Acquisition  (KA)  sessions.  Each  session 
focused  on  eliciting  knowledge  from  an 
AEW  expert  with  many  years  experience, 
and  with  knowledge  of  the  likely  evolution 
of  present  day  AEW  towards  FOAEW. 
During  this  phase  a  semi-structured 
interview  technique  was  employed  in  order 
to  provide  the  necessary  wide  breadth  of 
coverage  of  knowledge  required  for  the 
scoping  exercise.  All  sessions  were  recorded 
and  transcribed  (using  the  KA  toolkit 
PCPACK,  described  later)  providing  a 
permanent  record  of  the  interviews,  which 
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will  be  available  for  reference  throughout 
the  development  of  the  DSS  [1]. 

The  main  roles  for  maritime  forces  in 
FOAEW  were  identified  as  Surveillance, 
Attack  Co-ordination,  Airmanship 
(including  self-defence  and  safety)  and 
Communications.  Tasks  in  support  of  these 
capabilities  include  radar  and  sensor 
handling,  detection,  situation  assessment, 
tactical  decision  making,  fighter  control, 
reporting  and  airmanship.  For  each  of  these 
tasks  different  levels  of  support  were 
identified,  ranging  from  routine  activities  to 
those  requiring  active  intelligent  processing. 
Eleven  possible  areas  of  support  for  the 
FOAEW  tasks  were  identified  (see  figure  2). 
These  provide  the  sub-division  of  the  major 
functions  likely  to  be  involved  in  performing 
FOAEW.  For  each  of  these  areas,  specific 
tasks  were  identified  in  which  a  DSS  could 
provide  operator  support. 


Area 

Tasks 

Barrier 

Positioning 

Advice  on  optimal  barrier  position 
and  orientation. 

Advice  on  optimal  barrier  height. 

CAP 

Control 

Advice  on  optimal  CAP  position. 
Advice  on  optimal  CAP  height. 
Advice  on  optimal  CAP 
orientation. 

CAP  management  (e.g.  re-fuelling, 
order  of  replacements). 

Radar- 

Sensor 

Handling 

Advice  on  optimal  radar  scan 
direction,  power,  modes  and 
switches. 

Advice  on  anti-jamming  strategies. 
Advice  on  ESM  management. 
Advice  on  IR  scan  direction. 

Tactical 

Decision 

Making 

Advice  on  optimal  search  sectors. 
Automatic  prioritisation  of  in¬ 
bounds  via  the  threat  posed. 
Generation  and  monitoring  of 
expected  behaviour  patterns  and 
tracks  (e.g.  predicting  intercepts 
well  in  advance  by  automatic 
generation  of  expected  kill  lines 
and  weapon  release  lines). 

Monitoring  of  position  relative  to 
safe  lanes  (for  assessment  of  when 
and  in  which  direction  to  return  to 
force). 

Prosecutio 
n  and 

Attack  Co¬ 
ordination 

Automatic  transfer  of  attack 
orders. 

Calculation  of  predicted  intercept 
positions. 

At  intercept,  display  direction 
where  CAP  going  next. 

Reporting 

and 

Datalink 

Manage¬ 

ment 

Aid  in  the  efficient  use  of  datalink 
resources  available  to  FOAEW 
(i.e.  overcoming  bandwidth 
limitation)  for  transmitting  high- 
quality  information. 

The  intelligent  filtering  of 
incoming  data 

Database 

Manage¬ 

ment 

Search  and  pattern  matching  of 
mission  information  (Air  Tasking 
Order,  OPGEN,  OPT  ASK). 

Search  and  pattern  matching  of 
emitter  information  (ESM  data 
library). 

Search  and  pattern  matching  of 
intelligence  information. 

Airman¬ 

ship 

1 

Monitoring  of  fuel. 

Monitoring  of  position  relative  to 
mother  ship. 

Monitoring  of  changes  in  weather. 
Monitoring  of  fighter  and  weapon 
states. 

The  alerting  of  operators  to 
possible  safety  hazards. 

Self- 
defence 
and  Safety 

Display  of  appropriate  positions  of 
mission  engagement  zones  (MEZs) 
around  surface  contacts. 

Monitoring  of  position  relative  to 
surface  MEZs. 

The  alerting  of  operators  to 
possible  MEZ  encroachments. 

Secondary 

Roles 

Route  planning  for  probe,  attack 
support  and  OTHT  (i.e.  ASuW). 
Advice  on  search  and  rescue 
(SAR). 

Figure  2:  Areas  of  Intelligent  Support 


3  DESIGN  METHODOLOGY 
AND  KA  TOOLS 

Although  the  initial  scoping  phase  utilised 
semi-formal  interview  techniques,  the 
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current  design  phase  is  employing  a  more 
formal  approach  using  a  proven 
methodology,  REKAP  (REal-time 
Knowledge  Acquisition  Procedure),  and  a 
knowledge  acquisition  (KA)  toolkit, 
PCPACK,  which  is  a  practical  realisation  of 
the  methodology. 

3.1  REKAP  (Real-time  Knowledge 
Acquisition  Procedure)  Method 

REKAP  is  a  set  of  guidelines  for  structuring 
knowledge  acquisition.  REKAP  makes  the 
development  of  knowledge-based  systems 
more  efficient  by  helping  reduce  contact 
time  required  with  experts.  It  allows  a  rapid 
transition  from  the  description  of  the 
FOAEW  ‘knowledge  model’  to  the 
implementation  of  the  DSS  in  an  appropriate 
AI  programming  environment. 

REKAP  uses  the  KADS  [2]  four-layer 
model  of  expertise  as  its  framework  for 
integrating  the  knowledge  acquisition 
techniques  with  more  conventional 
techniques  of  real-time  structured  analysis. 
The  four  KADS  layers  are: 

•  the  Domain  Layer  describing  the 
structure  of  objects  in  the  application: 
for  example,  the  typical  speeds  of 
different  aircraft; 

•  the  Inference  Layer  outlines  the  basic 
problem-solving  steps  and  reactions  to 
events:  for  example,  the  use  of  new 
sensor  data  to  confirm  the  identity  of  an 
aircraft; 

•  the  Task  Layer  describes  the 
composition  of  tasks  and  system 
processes:  for  example,  the  breakdown 
of  actions  involved  in  the  generation  of 
route  plans;  and 

•  the  Strategy  Layer  indicates  task 
priorities  and  the  interactions  between 
processes:  for  example,  monitoring 
helicopter  status  while  identifying 
aircraft  and  planning  routes. 


REKAP  provides  the  knowledge  engineer, 

and  application  developer,  with: 

•  a  structured  method  for  building  real¬ 
time  knowledge-based  systems 
efficiently,  using  knowledge  acquisition 
(KA)  and  traditional  system  engineering 
(SE)  techniques; 

•  a  unified  framework  for  requirements 
definition,  knowledge  acquisition  and 
application  design; 

•  a  set  of  instructions  for  using  knowledge 
acquisition  techniques  and  tools  to  elicit 
and  structure  information;  and 

•  a  set  of  instructions  for  translating  a 
knowledge  model  into  an  application 
program. 


This  relationship  between  KA  techniques, 
knowledge  concepts  and  implementation 
structures  is  shown  in  the  table  of  figure  3. 


Level 
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REKAP 

(Framework) 

r/tUSE  Constritcls 
Clarysl) 
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Card  Son 
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Data  Slnjcture  Delinition 

Inference  Elemeni  Generator 
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INFERENCE 
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RepGrid,  Induction.  MatnxTool 
Process  Spedficeiion 

Control  S^dfication 

Inference  Complex 

Primitive  Task 

Rule 

lyo  Handler.  Demon 
Procedure 

Method 

TASK 
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Data  Flow 

Process  Decomposition 
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Ruloset,  Noticeboard 
Knowledge  Source 
Agent 

STRATEGY 
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Card  Sort 

Control  Flow 

Task  Priority 

Task  Interaction 

Task  Interruptibility 

Event  Priorities 
interrupt  Structure 
Inter-agent  Control 

Figure  3:  REKAP  Framework. 


The  process  of  applying  the  methodology  to 
a  particular  domain  is  shown  in  figure  4. 
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KA  Toolkit:  PCPACK 


Figure  4:  Use  of  REKAP 

The  information  derived,  through  the 
application  of  these  methods  and  techniques, 
includes: 

•  design  documents,  including  annotated 
data  flow  diagrams,  generated  as  a 
result  of  applying  the  methodology 
during  KA  sessions  with  domain 
experts,  and 

•  skeletal  code  definitions,  including 
object  class  definitions  and  the 
identification  of  program  modules,  from 
the  translation  of  the  tool  data, 
generated  from  the  KA  sessions. 

3.2  The  Generalised  Directive  Model 
(GDM) 

An  important  part  of  the  design  process 
using  REKAP  is  the  identification  of  the 
GDMs  (Generalised  Directive  Models),  for 
the  tasks  of  the  application. 

The  GDM  comprises  a  statement  of  the 
inference  steps  performed  during  problem 
solving  (for  example,  abstract,  match  and 
refine),  the  classes  of  domain  descriptions 
serving  different  problem  solving  roles 
(PSR)  within  the  model  (for  example 
observables,  variables,  abstract  solutions  and 
specific  solutions)  together  with  the 
dependencies  between  the  two. 

These  inference  steps  and  roles  correspond 
to  partitions  in  the  Imowledge  of  the  system. 
Aiding  acquisition  and  the  organisation  or 
indexing  of  the  knowledge  required  for  the 
system  to  function. 

For  example,  a  directive  model  for 
performing  situation  assessment  could  be  to 
match  the  known  features  of  a  particular 
contact  (track)  with  typical  descriptions  of 
certain  types  of  objects  (schemata). 


3.3 

Although  not  essential  to  the  design  process, 
significant  benefits  accme  from  using  an 
appropriate  toolkit  in  support  of  KA.  For 
the  FOAEW  DSS  knowledge  acquisition  is 
being  supported  by  the  KA  toolkit  PCPACK 
[3].  PCPACK  consists  of  an  extensive 
collection  of  KA  tools,  KADS  style 
directive  models,  and  facilities  for 
supporting  project  management  and 
documentation.  These  include: 

•  GDM  Workbench; 

•  Protocol  Editor  (or  Transcript  Analysis 
tool); 

•  Hyperpic  tool; 

•  Laddering  tool; 

•  Matrix  tool; 

•  Card  Sort  tool; 

•  Repertory  Grid  tool; 

•  Case  Editor  and  Rule  Induction  tool; 

•  Rule  Editor; 

•  Project  Management;  and 

•  Project  Documentation. 

The  KA  toolkit  binds  the  various  tools 
together  using  an  object  database,  within 
which  the  instances  of  concepts  derived  by 
each  tool,  and  their  inter-relationships,  are 
stored.  The  integrated  database  allows  the 
different  facets  of  the  stored  knowledge  to 
be  viewed  and  manipulated  by  other  tools  in 
the  toolkit. 


3.4  Estimating  Complexity 

The  methodology  provides  developers  with 
the  necessary  metrics  in  which  the 
complexity  of  the  application  can  be 
assessed  from  the  design,  and  thereby 
building  confidence  in  the  development 
process. 

A  earlier  decision  support  system  for  Anti¬ 
surface  Warfare  (ASuW)  provides 
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supporting  evidence  for  this  claim.  This 
application  was  developed  according  to  the 
same  methodology,  but  utilised  an  earlier 
KA  toolkit  (PROTOKEW)  together  with  a 
translation  tool  (PKTOMUSE)  capable  of 
translating  the  knowledge  model  directly 
into  the  structures  of  language  used  in  its 
implementation  (according  to  the  mapping 
described  in  the  table  of  figure  4,  above). 
In  the  situation  assessment  task  for  this 
system  virtually  all  of  the  schemas  (class 
definitions)  were  provided  by  the  application 
of  the  translation  tool  to  the  knowledge 
model,  while  the  task  decomposition  was 
able  to  identify  and  detail  each  of  the 
system’s  inference  stages. 

However,  the  accuracy  of  estimating  the  size 
and  complexity  of  a  task  is  very  much 
dependent  on  nature  of  the  task. 

For  example,  past  experience  has 
demonstrated  that  problems  involving 
certain  types  geometric  reasoning  are  more 
difficult  to  assess,  because  the  methods 
employed  by  the  human  operator  cannot  be 
easily  captured  in  a  way  that  can  be 
implemented  on  a  computer  (e.g.  the 
assessment  of  the  quality  of  flight  plans 
appears  to  easy  for  an  experience  operator  to 
perform,  but  is  extremely  difficult  for  a 
computer  based  solution). 

Since  a  similar  design  approach  is  being 
used  for  the  FOAEW  DSS,  it  is  expected 
that  similar  outputs  will  be  available  from 
the  process. 

4  DETAILED  KA  FOR  FOAEW 

All  of  the  KA  carried  out  during  the  detailed 
design  phase  for  the  FOAEW  DSS  is  being 
captured  using  the  PCPACK  toolkit,  which 
will  be  used  to  provide  a  knowledge  model 
for  each  of  the  tasks  identified  during  the 
scoping  phase. 


An  example,  taken  from  one  of  the  FOAEW 
tasks  nearing  completion  is  Barrier 
Positioning.  Figures  5  and  6  show  the  first 
level  description  for  this  task,  and  the 
decomposition  of  one  of  its  sub-tasks. 


Figure  5:  Barrier  Positioning  Control 


Figure  6:  Decomposition  of  Propose 
Horizontal  Position 


Abstracting  out  the  decomposition  provides 
us  with  an  appropriate  GDM  for  the  task. 
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which  is  identified  as  an  instance  of  a 
compare-select-revise  model  (figure  7). 


Figure  7:  GDM  Identified  for  Barrier 
Positioning 

The  design  process  is  often  found  to  be 
iterative,  since  the  GDM  may  be  unknown 
until  an  initial  examination  of  the  task  has 
been  carried  out.  Subsequently,  an 
appropriate  GDM  may  be  identified  but  it  is 
found  that  it  does  not  completely  match  the 
task  decomposition.  In  this  case  it  may  be 
desirable  to  reassess  the  task  to  see  if  it  can 
be  recast  to  match  the  GDM  more  closely. 


5  IMPLEMENTATION 
TOOLKIT:  D-MUSE 

The  proposed  implementation  framework 
for  the  FOAEW  DSS  is  D-Muse  [4],  a  real¬ 
time  knowledge  based  toolkit  for  the 
development  of  distributed  applications. 

D-Muse  provides  facilities  for  running  a 
collection  of  named  processes,  that  are 
inter-connected  by  a  network  of  deadlock 
free  communication  paths.  Each  Muse 
process  provides  the  following  functionality, 
appropriate  to  the  structures  required  by  the 


REKAP  method; 

•  flexible  knowledge  representation: 
including  forward  chaining  production 
rules,  backward  chaining  rules  and 
frames; 

•  support  for  modular  code  development: 
by  segmenting  domain  code  into 
separate  knowledge  sources  and  shared 
databases,  such  as  blackboard,  rule  sets 
and  notice  boards; 

•  support  for  real-time  operation: 

including  agenda-based  priority 
scheduling,  interrupt  handling  and  data 
capture,  and  the  D-Muse 

communication  facilities;  and 

•  extensible:  many  features  of  D-Muse 
can  be  customised  using  the  built-in 
object-orientated  language  (PopTalk)  or 
through  a  ‘C’  level  interface  allowing 
specialised  reasoning  structures  or 
access  to  any  external  software 
accessible  from  ‘C’. 

•  distributed  object  management: 
provides  the  mechanism  to  share 
information  between  Muse  processes  in 
real-time  while  ensuring  consistency 
(streamed  and  mirrored  objects). 

•  session  management  allows  the  creation 
and  deletion  of  D-Muse  processes  at 
run-time. 

The  core  knowledge-based  features  of  a 

Muse  process  are  depicted  in  the  following 

diagram,  figure  8. 
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Figure  8:  D-Muse  Process  Architecture 
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While  the  D-Muse  extensions  are 
represented  in  figure  9. 
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Figure  9:  D-Muse  Process  Architecture 

D-Muse  is  not  restricted  to  inter-connecting 
Muse  processes,  but  can  also  be  used  to 
communicate  with  other  processes,  such  as 
graphical  user  interfaces  and  simulation 
facilities. 

6  ARCHITECTURAL  DESIGN 

Another  important  goal  for  the  FOAEW 
DSS  is  that  it  should  provide  an  extensible 
implementation  that  supports,  where 
possible,  the  reuse  of  components,  and  the 
ability  to  grow  the  system  without  having  to 
re-engineer  components.  To  achieved  this, 
an  agent  based  architecture  has  been  chosen, 
which  is  being  implemented  within  D-Muse. 

The  internal  architecture  for  the  agent  is 
being  designed  so  that  they  can  be  populated 
directly  from  the  knowledge  models  created 
during  the  design  phase.  Figure  10  shows 
the  proposed  agent  structure. 


Figure  10:  Agent  Architecture 

Excluding  the  modules  responsible  for  the 
input  and  output  of  messages,  the  agent  has 
four  distinct  components; 

•  GDM  based  problem  solver,  which 
implements  the  main  task  of  the  agent 
(e.g.  route  planning,  situation  assessment 
and  barrier  positioning). 

•  World  Model,  which  provides  the  agent 
with  its  current  understanding  of  the 
world  (in  our  case  the  FOAEW  sortie) 
(including  general  principles,  state 
descriptions,  predictions  and  facts). 

•  Internal  Model,  this  defines  the  agents 
understanding  of  its  own  behaviour 
(including  beliefs,  principles,  objectives, 
goals,  intentions,  plans,  capabilities  and 
language/ontology). 

•  Acquaintance  Models,  representing  the 
agents  understanding  of  other  agents  it 
knows  about  (including  objectives, 
capabilities  and  language/ontology). 

Analysis  of  the  task  decomposition  and  its 
GDM  will  identify  the  requirements  and 
capabilities  of  the  agent.  In  particular, 
elements  of  the  world  model  will  be  defined 
by  the  input  and  output  parameters  to  the 
GDM.  Requirements  and  capabilities  will  be 
broadcast  (handled  by  a  facilitator  agent)  to 
the  other  agents  of  the  system.  The  other 
agents  are  able  to  respond  with  offers  of 
appropriate  information,  or  with  requests  to 
utilise  the  capabilities  offered  by  the  agent. 

For  example,  the  identification  of  MEZs, 
important  for  safe  route  planning,  can  come 
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from  many  different  sources,  some  of  which 
may  not  be  implemented  in  the  initial  DSS. 
The  addition  of  further  agents  that  expand 
the  scope  of  MEZ  identification  is  facilitated 
by  the  autonomy  offered  in  an  agent 
architecture. 


The  following  diagram,  figure  11,  shows  our 
initial  application  architecture. 


Figure  11:  Key  Software  Components 

The  agents  on  the  right  represent  the  kernel 
functionality  of  the  system,  the  central  set  of 
agents  define  the  support  agents  that 
interface  with,  in  our  DSS,  a  simulator  and  a 
workstation  based  operator  interface.  This 
decomposition  is  not  definitive.  Changes 
are  already  being  considered  to  partition  a 
number  of  the  agents  into  smaller  agents, 
with  more  specific  responsibilities,  which 
then  report  to  a  controlling  managing  agents. 

6.1  Agent  Communication  Language 

The  agent  communication  language  chosen 
for  the  FOAEW  DSS  is  the  Knowledge 
Query  and  Manipulation  Language 
(KQML).  KQML  [5]  is  a  language 
independent  message  format  and  message¬ 
handling  protocol  designed  to  support  run¬ 


time  knowledge  sharing  among  agents. 

The  KQML  language  defines  a  set  of 
messages,  or  “performatives”,  that  can  be 
used  for  communication  of  a  broad 
collection  of  broadcasts,  queries  and  replies 
between  agents.  These  performatives 
provide  agents  with  an  ability  to: 

•  advertise  their  capabilities, 

•  request  information  from  specific  or 
unknown  agents, 

•  dispatch  information  in  response  to  other 
agents’  requests,  and 

•  request  agents  to  modify  their  world 
model. 

Due  to  the  language  independence  of 
KQML,  the  actual  content  of  messages  is 
opaque  to  the  KQML  layer  (unless  the 
content  is  itself  KQML),  and  requires  the 
message  to  be  supplemented  with  attributes 
defining  the  language  it  is  expressed  in: 
Prolog,  Lisp,  PopTalk,  etc.,  as  well  as  the 
ontology  (i.e.  context)  it  assumes. 

However,  for  Muse  implemented  agents 
(this  will  apply  to  most  of  the  agents  within 
the  DSS),  only  the  semantics  of  KQML  will 
be  implemented.  The  syntactic  sugar 
surrounding  the  message  will  be  removed 
and  the  messages  communicated  between 
agents  as  Muse  objects.  Where  agents  are  on 
different  processors,  requiring  inter¬ 
processor  communication,  D-Muse  streamed 
objects  will  be  used.  This  approach, 
acceptable  within  a  Muse  based 
homogeneous  implementation,  significantly 
improves  the  efficiency  of  the  system, 
because  there  is  no  need  to  generate  and 
parse  the  KQML  syntax. 

However,  where  there  is  a  need  to 
communicate  with  agents  that  have  not 
implemented  in  the  Muse  language,  or  do 
not  support  D-Muse  streamed  object 
communication,  the  full  syntactic  form  of 
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KQML  messages  will  be  employed. 

To  make  the  messaging  transparent,  the 
system  will  use  the  acquaintance  models 
built  up  by  the  agents  of  the  system  to 
provide  an  agent  with  the  language  and 
ontology  required  by  the  agent  receiving  the 
communication. 

This  approach  will  aid  future  desires  to 
integrate  the  DSS  with  legacy  systems,  such 
as  intelligence  and  emitter  databases. 

6.2  Optimisation  of  Inter-agent 

Communication 

An  agent  defines  a  logical  partition  of 
functionality  within  the  application,  but  does 
not  require  that  it  be  localised  to  a  single 
processor.  A  proposal  is  currently  being 
investigated  of  ways  of  distributing  agent 
functionality  across  processor  boundaries  to 
find  ways  of  improving  efficiency  of  inter¬ 
agent  communication  in  a  distributed 
environment. 


For  example,  consider  the  following,  figure 
12,  where  agents  2,  3,  and  4  all  regularly 
request  information  firom  agent  1. 


Figure  12:  Inter-agent  communication 
without  optimisation 


If  each  agents  requires  the  same  information 
(i.e.  results  of  situation  assessment  applied 
to  an  ingressing  aircraft),  separate  inter¬ 


processor  messages  require  to  be  generated 
for  each  agent. 

The  proposal  is  to  define  a  “proxy”  for  the 
agent  which  can  reside  on  the  same 
processor  as  the  agents  requesting  the 
information.  All  messages  to  agent  1  are 
first  directed  to  the  proxy,  which  will 
attempt  to  satisfy  the  request  locally.  If  the 
request  cannot  be  satisfied  locally  the 
message  is  redirected  to  processor 
containing  the  agent  proper.  This  is  depicted 
in  figure  13. 

In  terms  of  its  implementation  it  is 
envisaged  that  this  will  involve  sharing 
elements  of  the  world  and  acquaintance 
models  across  processor  boundaries 
(implemented  using  D-Muse  mirrored 
objects). 


Figure  13:  Inter-agent  communication 
with  optimisation 


6.3  Fault  Tolerance  and  Agent 
Mobility 

Since  the  proposed  D-Muse  agents  can  be 
created  dynamically  across  any  of  the 
available  CPUs,  the  run-time  system 
provides  capability  for  implementing  a 
limited  degree  of  fault  tolerance  within  the 
DSS.  For  example,  if  a  processor  dies  it  is 
possible  to  terminate  the  function  of  low 
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priority  tasks,  and  use  the  freed  resources  to 
host  new  instances  of  agents  from  the  lost 
processor. 

Although  it  is  technically  feasible,  there  are 
many  issues  relating  to  acceptable  grade  of 
service  and  system  consistency  to  be 
addressed. 

D-Muse  agents  also  provide  the  ability  to 
support  mobile  agents,  which  can  be  moved 
(intact)  at  run-time.  However,  a  need  for 
such  capability  within  the  FOAEW  DSS  as 
yet  to  be  identified. 

7  CURRENT  STATUS  OF  THE 
FOAEW  DSS 

At  present,  the  project  is  entering  the  final 
stages  of  its  design  phase,  with  knowledge 
elicitation  for  a  number  of  tasks  still 
ongoing.  As  such  the  information  presented 
here  is  still  subject  to  change.  However,  it  is 
intended  that  the  philosophy  of  the  design, 
should  be  retained. 

An  experimental  general  agent  framework 
has  been  built  and  is  being  used  to  refine  the 
ideas  for  a  practical  agent  structure  for  the 
application,  which  will  be  optimised  for  the 
application.  Ways  of  importing  the  tasks,  via 
their  GDM,  into  the  agent  architecture,  and 
addressing  issues  of  maintenance  of  the 
world  model  are  also  being  addressed. 

Although  a  translation  tool  exists  for  the 
earlier  KA  tool  (PROTOKEW)  to  Muse,  the 
present  toolkit  does  not  yet  have  this  facility. 
However,  Epistemics  are  investigating  the 
possibility  of  providing  such  a  tool,  and  it  is 
hoped  that  by  the  time  implementation  of 
the  DSS  begins  this  shortcoming  will  have 
been  addressed. 

8  CONCLUSION 

The  methodology,  design  tools  and  the 


proposed  agent  architecture  for  the  FOAEW 
DSS  provides  the  developers  with 
confidence  in  the  development  of  the 
application. 

In  particular,  the  close  relationship  between 
the  methodology,  it  realisation  in  the 
PCPACK  toolkit  and  the  ability  to  map  the 
knowledge  concepts  directly  onto  features  of 
the  implementation  language,  provide  a 
means  of  estimating  the  size  of  the 
application  (including  the  size  and  quantity 
of  data  struetures,  and  the  complexity  of  the 
various  functional  areas  and  tasks  of  the 
application). 

The  utilisation  of  an  agent  architecture, 
although  grounded  more  in  practical 
implementation  rather  than  a  theoretical 
implementation,  provides  the  developers 
with  means  of  reusing  software  components 
and  supporting  future  modular  expansion. 

The  use  of  KQML  style  communication 
between  agents  of  the  application,  allowing 
easily  transition  into  the  KQML  syntax  for 
communication  with  external  data  sources 
that  will  grow  in  importance  as  the  system 
expands. 
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I.  INTRODUCTION 

Since  1994,  the  Technical  Service  for  Aeronautical 
Telecommunication  and  Equipment  of  French  DGA 
(DCAE/STTE  then  DSA/SPAE)  launched  an 
exploratory  development  program  concerning  a  high 
level  decision  aid,  using  Knowledge  Based  System 
(KBS)  technology  for  an  advanced  combat  Aircraft. 
This  french  project  for  an  in  flight  mission  replanning 
Decision  Aid  is  called  "Copilote  Electronique" 
[Champigneux  1 989] . 


The  exploratory  development  program  is  led  by 
Dassault  Aviation  with  the  support  of  many  industrial 
and  scientific  partners  (SAGEM,  Dassault  Electronique, 
Matra  BAe  Dynamics,  Sextant  Avionique,  IMASSA, 
ONERA...). 

It  aims  at  introducing  this  kind  of  decision  aid  within  a 
2010  horizon  for  a  future  Rafale  standard  (The  Rafale 
aircraft  will  enter  the  French  Air  Forces  at  the  start  of 
the  next  century). 
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I.  OPERATIONAL 
OBJECTIVES 

•  The  operational  objectives  of  the  Copilote 
Electronique  are  surveyed  in  order  to  precise  the 
domains  of  assistance  that  are  relevant  for  such  a 
system. 

Before  the  launch  of  the  Exploratory  Development  an 
action  was  initiated  by  DRET  the  french  military 
research  agency  to  survey  the  need  for  pilot  assistance 
in  french  airforce  programs  and  the  feasibility  of  KBS 
as  a  potential  technical  answer  (DRET  86.34407 
faisabilite  d'un  Copilote  Electronique  and  DRET 
89.34565  Methodes  permettant  le  developpement  d'un 
Copilote  Electronique).  The  need  was  expressed  by 
senior  pilots  of  the  french  airforce  and  navy,  with 
experience  of  Mirage  FI,  Mirage  2000  and  Super 
Etendard.  The  cognitive  analysis  of  pilots  activities  was 
conducted  by  CERMA  (Centre  for  Medical  studies  and 
research  in  Aerospace). 

Within  the  context  of  the  Copilote  Electronique 
program  this  initial  survey  was  completed  and  reviewed 
in  front  of  the  forecasted  definition  of  the  Rafale 
missions  and  system  standards.  This  involved  a 
specialist  of  the  Rafale  program  from  the  CEAM 
(French  air  force  test  center)  as  well  as  Dassault  test 
pilots  currently  involved  in  the  definition  of  the  new 
program. 

This  section  do  not  address  specific  requirements  linked 
with  the  Rafale  program  but  the  generic  needs  of  an 
onboarded  mission  planning  activity  in  a  future  combat 
aircraft. 

Conducting  penetration  missions  in  hostile  territory  has 
always  raised  problems  of  workload  on  single  pilot. 
Regardless  of  aircraft  configuration  and  avionics  the 
planning  activity  is  a  very  difficult  task  for  the  pilot  in 
flight.  This  includes  route  selection,  ECM  employment 


(like  activating  and  shutting  down  jammers,  throwing 
decoys...),  flight  monitoring  (following  profile, 
respecting  timing,  handling  communication  with  C3I...), 
attack  planning  and  weapon  selection...  This  overload 
problem  has  generally  been  solved  by  applying  strict 
mission  control  rules  over  a  very  detailed  ground  based 
mission  preparation. 

It  is  recognised  for  example  that  in  a  typical  Penetration 
Mission  at  low  altitude  and  high  speed  within  enemy 
territory,  a  pilot  is  following  a  strict  time  schedule  with 
little  possibilities  to  divert  from  it.  For  instance  at  an 
altitude  of  300  to  500  feet  and  a  speed  of  500  knots 
only  a  few  seconds  of  delay  over  the  way  points  can  be 
accepted.  If  such  timing  is  not  followed  coordination 
between  friendly  ressources  is  in  danger  ,  the  efficiency 
of  weapon  delivery  is  lowered  and  possibly,  the  firing 
of  the  aircraft  by  friendly  ground  defense  will  happen 
when  crossing  back  the  Front  Edge  of  Battle  Area 
(FEBA). 

The  extreme  time  pressure  imposed  on  pilots  of  combat 
aircraft  makes  the  planning  activity  very  complex  and 
dynamic.  With  such  an  extreme  time  pressure,  in-flight 
planning  could  be  considered  as  totally  unrealistic,  but 
it  must  be  recognized  that  most  of  the  time  real  missions 
will  be  disturbed  by  unexpected  events.  This  leaves  no 
choice  to  the  pilot  who  needs  replanning. 

We  have  listed  many  of  those  unexpected  events  in  the 
domain  of  aircraft  ressources.  One  may  mention  engine 
failures,  jammed  positioning  systems,  sensor  default... 
They  are  also  numerous  and  frequent  in  the  tactical 
domain.  For  instance  one  will  encounter  hostile  Conter 
Air  Patrol  aircrafts  (CAP),  unknown  ground  missile 
sites  (SAM),  electronic  counter  measures...  There  are  of 
course  perturbations  due  to  the  natural  mission 
conditions  such  as  weather  evolutions,  unregistrered 
ground  obstacles...  Finally  one  have  to  mention 
coordination  problems  between  raid  and  escort  patrol  or 
Command  and  Control  aircraft  (AW ACS)  as  well  as 
possible  human  errors. 
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Therefore  a  strictly  nominal  execution  of  a  mission  plan 
prepared  on  ground  is  very  unlikely  to  happen 
according  to  our  experience  of  Mirage  FI  CR  or  Mirage 
2000  operations  as  well  as  Rafale  extrapolations.  Even 
simulation  campains  will  show  frequent  perturbations 
with  the  necessity  for  the  pilot  to  react  by  in  flight 
mission  planning.  A  recent  Rafale  simulated  test  of  a 
sweep  mission  exemplified  the  interest  of  some  pilot 
support  in  heavy  workload  situation. 

In  such  cases  the  functional  requirements  of  pilot 
planning  task  ranges  from  flying  activities,  navigation, 
ressources  management,  information  pick  up,  to  tactical 
response  elaboration. 

For  instance  we  analysed  in  detail  an  Air  to  Air 
Engagement  during  an  escort  mission.  In  case  of 
ennemy  engagement,  a  pilot  has  to  planify  an  adapted 
behavior  to  analyse  the  tactical  situation  including 
platform  manoeuver  and  sensor  control.  He  needs  to 
coordinate  the  friendly  actions  through  communication 
with  the  penetrating  raid  bomber  leader  and  with  its 
fighter  wingman.  He  exchanges  tactical  information. 


selects  tactics,  assigns  target  and  he  instantiates  a 
proper  offensive  plan  including  weapon  preparation, 
launch  point  calculations,  flight  path  trajectories 
generation,  evaluation  of  kill  and  survival  predictions... 

In  conclusion  of  this  section  we  can  assess  that  a 
requirement  for  in  flight  mission  planning  is  perceived 
in  future  low  altitude  high  speed  penetration  missions 
and  air  to  air  escort  missions  (this  is  not  to  say  that  in 
flight  planning  is  not  needed  on  other  missions  such  as 
air  to  air  interception  at  high  altitude  but  this  analysis 
was  not  carried  exhaustively  within  the  scope  of  our 
project). 

The  planning  task  not  only  concerns  navigation 
strategies  but  also  tactical  offensive  and  defensive 
management  as  well  as  aircraft  ressources  monitoring. 
These  many  planning  concerns  overlaps  during  active 
mission  phases  such  as  air  to  air  engagement. 
Consequences  of  bad  planification  as  taking  wrong 
decisions,  acting  too  late,  or  executing  improperly  the 
plan,  are  generally  intolerable.  It  may  result  in  a  crash. 
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or  an  unsuccessful  mission,  or  the  loss  of  aircraft  and 
pilot ... 

A  single  pilot  with  current  avionics  is  unlikely  to 
perform  such  in  flight  planning  complex  task  without 
errors.  A  need  for  assistance  is  perceived  ,  leading  to 
increased  autommation  as  well  as  planning  support.  We 
noticed  during  our  analysis  phase  of  the  Copilote 
Electronic  that  there  is  a  general  preference  for  systems 
providing  assistance  in  tasks  such  as  calculating  fuel, 
plotting  routes,  identifying  risks...  Pilots  really  care  for 
better  situation  assessment  in  the  planning  process.  This 
was  well  expressed  by  Major  G.W.  Breeschoten  in  his 
keynote  address  of  Guidance  and  Control  Panel  53rd 
Symposium  [Breeschoten  1992]_: 

"I  do  not  want  the  system  to  think  for  me,  at  least  in  the 
sense  that  it  prescribes  my  tactical  actions.  It  can  to 
some  extent  think  with  me.” 

Nevertheless,  as  critical  decisions  are  to  be  taken  on 
uncertain  or  tactical  aspects  of  mission,  aircraft 
designers  often  rely  on  pilots  judgement.  This  tendency 
is  even  currently  required  by  Air  Forces. 

As  a  result  of  this  requierement  analysis  the  Copilote 
Electronique  project  is  oriented  toward  a  multi-agent 
(or  multi-assistance)  organisation  that  best  express  the 
human  resonning  in  the  guidance  and  control  domain. 
For  the  development  phase  of  the  program  it  was  then 
decided  to  consider  the  expert  domains  that  pilots 
distinguish  in  the  conduct  of  penetration  and  escort 
mission. 

These  domains  are: 

Aircraft  system  management  («  domaine  Avion  ») 
including: 

-  System  Evaluation 

Monitoring  discrete  events  and  continuous 

signals. 

Assessing  real  avionic  systems  states  and 
dependability. 

-  System  planning 

.  Planning  the  avionic  systems 
reconfiguration. 

.  Scheduling  of  action  &  ressources  according 
to  the  plan. 

Tactical  management  («  domaine  Tactique  Sol  »  & 

«  domaine  Tactique  Air  ») 
including: 

-  Tactical  assessment 

.  Analysis  of  friendly  &  foe  forces. 

.  Elaboration  of  forecasted  evolutions. 

Assessment  of  risk/efficiency  according  to 

present  plan. 


-  Tactical  planning 

.  Planning  tactics  according  to  the  threats  and 
pilot  strategies. 

.  Scheduling  actions  and  ressources  according 
to  the  tactics. 

Handling  conflicts  among  proposed  tactics. 

Mission  management  («  domaine  Mission  ») 
including: 

-  Mission  condition  Assessment. 

.  Mapping  of  pre-mission  meteorological 
brieffing  onto  possible  routes. 

.  Mapping  of  pre-mission  geographical  data 
onto  possible  routes. 

-  Route  Planning 

.Selecting  re-routing  options  accoring  to  the 
updated  mission  context. 

.  Planning  new  routes. 

.  Monitoring  possible  routes  with  quality 
estimates. 

Man  Machine  coordination  («  domaine  Coherence 
des  assistances  ») 
including: 

-  Pilot  Behavior  Assessment. 

.  Mapping  pre-mission  strategic  option  to  in¬ 
flight  planning. 

Inferring  pilot  intent  from  observed  actions. 

-  Planning  management 

Driving  experts  planning  efforts  toward  a 
common  goal.in  accordance  with  pilot  strategy 
.  Insuring  proposed  plan  quality 
Dialogue  management. 

.  Presenting  relevent  informations  to  the  pilot 
.  Handling  pilot  querries 

I.  ERGONOMICAL  DESIGN 

•  The  advantages  of  a  cognitive  assistant  approach 
over  an  automatic  planing  approach  and  the 
ergonomical  rules  settled  by  the  project  to  facilitate 
in  flight  Pilot<->System  relationship  are  presented 
showing  the  "Copilote  Electronique"  orientations  in 
that  respect. 

To  design  the  proper  decision  aid  it  is  necessary  to 
analyse  the  level  of  autonomy  best  adapted  to  the  in 
flight  planning  process. 

In  front  of  the  increasing  complexity  of  avionic  systems 
and  weapon  systems  it  is  certainly  desirable  to  design 
systems  capable  of  taking  responsibilty  of  lots  of  pilot 
decision  activities.  The  present  technological  push,  best 
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exemplified  by  the  well  known  knowledge  based 
systems,  expert  systems,  constraint  programming  tools, 
neural  nets...  leaves  an  open  field  to  the  dream  of  full 
automation. 

Of  course  some  caution  should  be  taken  in  terms  of 
feasibility  for  these  techniques  in  real  time  avionics.  As 
Wiener  and  Curry  showed  [Wiener  1980],  fiill 
automation  can  have  serious  drawbacks  with  a  risk  in 
the  long  term  of  having  operators  unable  to  conduct  the 
missions. 

Various  embedded  functions,  such  as  navigation, 
piloting,  aircraft  status  management,  weapons  system 
management,  and  in  some  extension  sensors 
management  have  been  successfully  automated  by 
classical  software  engineering  methods,  but  the  addition 
of  such  separate  and  independent  automated  functions  is 
more  and  more  difficult  to  control  in  real  time  situations 
by  human  pilots. 

Automated  functions  are  intended  to  increase  in  number 
and  complexity,  in  the  foreseen  tactical  context  of  year 
2010.  Such  context  is  characterised  by  a  great  number 
of  various  possible  threats,  with  electronic  war  systems 
and  new  sophisticated  weapons.  Operational  experts 
think  that  future  pilots  will  have  some  difficulties  with 
this  combinatory  explosion  of  information  sources 
unless  being  assisted  in  their  reasoning  tasks. 

Within  the  "Copilote  Electronique”  project,  we 
cautiously  analysed  the  tasks  in  terms  of  potential  for 
automation  and/or  assistance.  We  based  our  first  work 
on  the  guidelines  of  the  AGARD  advisory  report  on 
improved  guidance  and  Control  for  the  automation  at 
the  Man  Machine  Interface  [AGARD  1988]. 

These  guidelines  expressed  that  tasks  requiring  highly 
accurate  responses,  fastidious  and  repetitive  actions, 
and  exhaustive  calculations  are  good  candidate  to 
automation.  On  the  other  hand,  tasks  requiring 
judgement,  multi-sensory  information  gathering, 
hypothetical  reasoning,  contingency  reaction...  are  best 
suited  for  a  "Man  in  the  loop"  design. 

Planning  tasks  are  certainly  of  the  second  type.  We 
structured  those  tasks  like  system  reconfiguration, 
ressources  scheduling,  navigation,  fuel  monitoring 
threat  analysis,  threat  avoidance,  threat  engagement. 
Command  Control  and  Communication,  sensor  control 
and  weapon  management  according  to  the  expert 
domains  : 

-  System  management 

-  Tactics  managment  (air-air  and  air-ground) 

-  Mission  management 


In  the  System  area  ,  planning  is  more  often  an 
optimization  of  fine  grain  plan  in  front  of  the  flight 
parameters  evolution  and  generalised  state  of  the 
navigation  and  weapon  systems  (including  faulty 
states). 

In  the  Tactics  area  ,  planning  is  reactive  .  Threats  are 
poping  up  as  unexpected  event  and  disrupt  from  the 
planified  behavior  established  on  ground  during  the 
preparation  phase. 

In  the  Mission  area  ,  the  result  of  the  mission 
preparation  remains  the  guide  for  all  in  flight  planning. 
The  task  here  consists  of  adaptations  of  the  nominal 
plan,  plan  refinement  in  a  precise  context,  choice  of 
alternative  plans... 

At  this  stage  of  our  design  we  have  oriented  our 
approach  toward  a  human  centered  design.  This  was 
based  on  human  factors  evidences  from  the  aviation 
history,  which  are  addressed  in  the  "Copilote 
Electronique"  team  by  IMASSA/CERMA  [Amalberti  et 
al  7992]. 

This  study  resulted  in  “user  oriented  rules”  that  has  to 
be  used  in  the  design  of  the  Copilote  Electronique. 

Those  rules  can  be  summarised  as  follows  : 

•  pilot  anticipates  and  needs  anticipation  assistance  on 
contrary  of  “classical  engineer  designed”  assistance 
which  are  often  too  reactive, 

•  pilot’s  decisions  reflect  often  compromises  between 
mental  load  and  ideal  response  to  the  situation,  so  pure 
optimality  is  not  to  be  researched  if  pilot  has  no 
sufficient  time  to  understand, 

•  following  their  own  personal  skills,  different  pilots 
may  organise  work  differently,  assistance  must  be 
adapted  to  these  skills, 

•  assistance  must  be  homogeneous,  and  it  will  be 
preferable  to  rely  on  specialised  expert  for  each 
operational  domain  (e.g.  Strike  or  Air  Defense 
expertise)  so  resulting  assistance  will  produce  constant 
understanding  interpretation  model  that  will  avoid 
surprises  for  pilot, 

•  assistance  must  know  and  respect  its  own  limits, 

•  system  design  may  use  “what  if’  approach  to  be  less 
reactive, 

•  dialogue  must  be  adapted  to  context,  pilot  intents  and 
pilot  load, 

•  dialogue  must  be  space  oriented  and  interactive,  better 
use  vocal  media  than  written,  but  avoid  saturation, 

•  respect  logic  of  pilot  understanding,  that  means  rely 
on  the  understanding  model  designed  with  expert  pilots. 
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The  french  Copilote  Electronique  project  is  oriented 
toward  a  cogitive  assistance  as  a  consequence  of  this 
ergonomical  analysis. 

I.  ORGANIC  ARCHITECTURE 

•  The  organic  architecture  established  for  the 

"Copilote  Electronique  and  the  proper  mechanisms 
supporting  the  cognitive  assistant  approach  of 
mission  planning  are  described. 

In  order  to  achieve  the  main  objective  of  demonstrating 
the  concept  of  a  cognitive  assistance  for  future  combat 
aircraft  it  is  necessary  to  organize  the  selected  expert 
domains  that  will  perform  the  required  functionalities  of 
in  flight  decision  aid. 

The  Copilote  Electronique  project  finalized  such  an 
architecture  by  the  end  of  1994. 

The  top  level  organization  of  the  expert  domains  in  the 
Copilote  Electronique  is  in  accordance  with  the 
Functional  Decomposition  of  Generic  decision  system 
in  Guidance  and  Control  as  proposed  by  AGARD 
Working  Group  11  [AGARD  1993]. 

The  two  main  activities  of  situation  assessment  and 
planning  are  represented  in  each  of  the  expert  domains. 
All  the  expert  domains  are  communicating  with  others 
to  enrich  their  vision  of  the  situation  and  to  elaborate 
plans.  The  coordination  activity  is  taken  in  charge  by  a 
specific  expert  supervising  the  others. 


An  expert  domain,  absorbs  high  rates  of  raw 
information,  select  and  highlight  the  more  crucial  ones, 
before  initiating  dialogue  with  the  other  experts.  Raw 
data  is  provided  by  the  existing  technical  functions  of 
the  Navigation  and  weapon  system  assuming  that  a  data 
sharing  mecanism  is  avalable  (it  is  the  case  with  Rafale 
and  M2000  type  of  system). 

The  planning  reasonning  layer  of  each  domain  take 
entries  from  the  assessment  level.  Expert  description  of 
the  situation  are  not  propagated  to  each  domain  but 
relevant  informations  can  be  accessed  on  request. 
Planning  directives  are  passed  by  the  supervising  expert 
to  the  concerned  specialists  according  to  the  problems 
encontered.  Such  directives  includes,  problem  scope, 
constraints,  and  pilot  strategies.  The  experts  reason  in  a 
manner  adapted  to  current  situation  and  mental  load  of 
the  pilot.  They  consider  a  restricted  set  of  actions 
choice  for  the  pilot  and  examine  all  consequences 
before  proposing  them. 

Dialog  with  the  Pilot  is  handled  at  the  supervision 
expert  level.  It  insures  that  a  single  coherent  proposal 
will  be  presented  by  the  group  of  expert  domains.  It 
also  minimize  the  informational  woarkload  of  the  pilot 
and  handles  the  pilot  queries  through  the  use  of 
«  regular  »  man  machine  interface  of  the  Rafale  aircraft. 

The  external  world  perception,  the  communication  with 
other  agents  and  the  plan  execution  are  not  part  of  the 
Copilote  Electronique  responsibility  but  we  can  assume 
that  these  activities  are  present  in  the  current  Navigation 
and  Weapon  system  (SNA)  in  which  the  Copilote 
Electronique  is  integrated . 
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Perception  Execution 

Navigation  &  Weapon  System 


The  dynamic  behavior  of  the  Copilote  Electronique  is 
driven  by  a  cyclic  assessment  of  the  situation  by  the 
expert  domains  (the  period  of  the  cycle  differing  from 
one  expert  to  another)  and  an  event  driven  planning 
activity  based  on  the  warnings  issued  by  the  assessment 
layer. 

The  planning  activity  includes  several  steps  of 
generation  driven  by  the  experts  best  suited  to  the 
revealed  problem  and  the  pilot  strategy.  Those  steps  are 
followed  by  qualification  treatment.  Qualification  is 
performed  by  all  the  expert  domains  so  that  the  quality 
of  the  proposed  solution  is  seen  globally  and  not  only 


by  a  single  domain  with  possible  conflicts  with  other 
fields.  In  case  of  insufficient  quality  constraints  are 
posted  by  the  expert  domain  to  help  the  other  in  refining 
their  proposal. 

Once  finished  the  planning  activity  gives  results  to  the 
dialog  manager.  Plans  are  joined  to  situational 
information  to  be  presented  to  the  pilot.  Two  levels  of 
dialog  are  handled  (rich  or  succint)  in  order  to  adapt  the 
information  flow  to  the  pilot  workload. 

The  following  figure  present  this  process: 


Data  from 
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& 
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Navigation  and  Weapon  System 
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I.  DEVELOPMENT  STATUS 

•  A  short  overview  of  the  knowledge  based 
development  process  engaged  is  given 

The  goal  of  the  functional  development,  launched  in 
1994  for  a  three  years  duration,  is  a  ground  simulation, 
without  real  time  constraints,  to  illustrate  the  potential 
of  the  “Copilote  Electronique”  in  situation  of  strike  and 
escort  missions,  with  low  altitude  penetration 
constraints. 

The  software  architecture  at  this  stage  is  resolutely  a 
cooperative  set  of  expert  modules  mapping  the  expert 
domains  [Gilles  1991]. 

To  conduct  this  development  Dassault  Aviation  set  up  a 
consortium  based  on  the  french  industrial  competences. 
Responsibilities  within  the  consortium  are  : 

•Ergonomics  rules  and  knowledge  acquisition 
methods  and  verification  tasks 
->  IMASSA/CERMA 

•System  Status  Assessment  and  Management 
->  SAGEM 

•Tactical  Situation  Assessment  and  Management 
(Ground  threat  and  defensive  Counter 
Measures)  ->  DASSAULT 
ELECTRONIQUE 

•Tactical  Situation  Assessment  and  Management 
(Air  threat  and  offensive  Weapons) 

->  MATRA  DEFENSE 

•Mission  Conditions  Assessment 
and  Mission  Management 
->  SEXTANT  AVIONIQUE 

•Pilot  Assessment, 

action  plans  assessment, 
relevant  information  management 
and  man  machine  interface 
->  DASSAULT  AVIATION 

Knowledge  engineering  techniques  are  for  expertise 
initial  design.  With  IMASSA,  a  specific  method  for 
eliciting  and  formalising  pilot’s  expert  knowledge  was 
studied  and  is  used.  It  is  supported  by  a  formalisation 
tool  called  X-PERT.  It  is  confirmed  by  present 
campains  that  pilot  expertise  can  be  collected 
coherently  in  all  the  expert  domains  and  that  generic 


behaviors  (not  linked  with  a  specific  Navigation  and 
weapon  system)  can  be  used  in  the  expert  modules. 
Generic  expertise  has  to  be  supplemented  by  extensive 
knowledge  evaluation  and  correction  in  simulator,  in 
order  to  represent  specific  behaviors  linked  with  the 
new  system  like  Rafale.  The  main  issue  for  the  future 
design  is  to  accept  expertise  from  pilot  during 
operational  life  of  the  system. 

The  technical  specification  is  driven  toward  a  flexible 
heterogenous  implementation  paradigm.  The  Copilote 
Electronique  expert  modules  are  organised  in  a  multi¬ 
agent  system  using  Distributed  Artificial  Intelligence 
techniques  [Erceau  1991]. 

Another  very  important  technical  issue  is  the  definition 
of  a  common  “plans  and  goals”  exchange  language 
between  all  specific  assistance  modules,  and  great 
efforts  are  made  to  maintain  this  common  message 
glossary.  Within  the  functional  development  Dassault 
Aviation  proposed  an  exchange  language  called  LDI 
which  provides  a  CORE  A  like  facility  for  object 
communication. 

A  unifying  technical  principle  is  adopted  to  facilitate 
the  architecture  design  via  the  intent  planning 
paradigm.  This  principle  is  essential  to  fulfil  general 
ergonomics  constraints  :  assistance  must  not  participate 
to  the  signalled  existing  overloading  factors.  Intent 
recognition  is  a  challenging  but  promising  direction  and 
can  be  made  easier  by  extended  preparation  mission 
plans  and  procedures  (for  each  pilot  activity)  that  will 
be  perhaps  the  new  “automated  and  personalised”  check 
lists  version  of  the  future  [Rouse  1991]. 

At  present,  a  mock-up  is  implemented.  It  uses  a  set  of 
Unix  workstations  (one  for  each  expert  domain)  linked 
to  a  Rafale  simulator  with  «  engineer  » type  of  interface. 
The  mock-up  shows  non  real  time  behavior  of  the 
expert  modules  integrated  in  a  complete  Copilote 
Electronic  system.  A  synthesis  of  the  presented 
functionalities  will  be  realised  in  spring  1997. 

I.  CONCLUSION 

This  example  opens  to  the  possible  future  developments 
of  intelligent  decision  aids  for  in  flight  tactical  dynamic 
planning  within  future  combat  aircraft. 

The  technology  is  available  today  to  provide  viable 
knowledge  system  solutions  to  well-chosen  and  well- 
defined  problems.  It  can  be  expected  to  see  more  and 
more  successful  projects  on  such  on-board  applications. 
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as  both  the  research,  the  technology  and  engineering 
skills  of  application  developers  improve. 

But  this  process  may  be  slower  than  was  though.  Main 
reason  is  that  knowledge  acquisition  tasks  and  user 
oriented  ergonomics  rules  compliance  must  be 
integrated  in  the  overall  engineering  cycle. 

The  french  Copilote  Electronique  project  has  been 
carefully  planified  considering  those  methodological 
difficulties. 

After  a  long  design  phase  the  Copilote  Electronique  is 
now  in  a  software  development  phase.  The  planning 
domains  are  the  main  drivers  of  this  development.  They 
are  developped  by  french  industrial  partners  in  a 
federative  approach.  Each  partner  brings  to  the  project  a 
specific  background,  with  a  high  value  knowledge  of  his 
planning  field  and  mastering  of  appropriate  planning 
mecanisms.  This  results  in  a  very  rich  but 
heterogeneous  multi-expert,  multi-industrial  dynamic 
planning  system. 

The  Copilote  Electronique,  not  only  reach  a  successful 
behavior  in  each  dynamic  planning  field,  but  also 
achieves  to  demonstrate  a  coherent  assistance  for  in 
flight  decision  aid.  Special  care  is  taken  to  analyse 
interdependancies  between  the  various  plans  and  to 
respect  the  rules  of  a  good  man  machine  relationship. 
Expert  pilots  give  feedback  on  the  quality  and 
acceptability  of  the  resulting  planning  assistant. 
According  to  their  remarks  the  architecture,  mecanisms 
and  knowledge  of  the  Copilote  Electronique  planners 
can  be  tuned.  Present  scenarios  give  confidence  on  the 
resulting  operational  benefits  of  the  assistance  system. 

Dynamic  replanning  proposals  will  be  demonstrated  on 
a  realistic  full  mission  simulator  after  optimisation  of 
the  present  mock-up.  Real  time  performances  of  the 
resulting  planning  system  will  be  optimised  with  the 
help  of  current  technological  progress  :  a  better 
measurement  of  real-time  constraints  for  which,  for  the 
moment,  only  provisional  precautions  have  been  taken 
into  account  in  the  supervisor  architecture  mecanisms 
(for  example  by  adopting  "any  time"  algorithm 
methodology  as  in  [Thierry  SALVANT 1997]) .  But  our 
belief  is  that  the  key  of  a  successftil  in  flight  replanning 
assistance  is  to  keep  an  approach  equally  dependant  on 
the  pilots  cognitive  abilities  acquisition  and  on 
hardware/software  evolution. 

The  first  steps  of  the  Exploratory  Development  phase 
confirms  that  the  distributed  architecture  and  the 
Human  driven  design  approach  are  good  drivers  for 
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1.  Summary 

Clumsy  automation  is  considered  to  be  a  major  reason 
for  deficiencies  concerning  the  interaction  between 
cockpit  crew  and  aircraft  systems.  Cockpit  assistant 
systems  are  being  developed  in  support  of  human- 
centered  automation. 

In  this  paper  a  general  survey  on  the  principals  of  cockpit 
assistance  will  be  given.  Demands  and  requirements  for 
an  appropriate  automation  and  a  functional  structure  of 
an  knowledge-based  assistant  system  will  be  introduced, 
leading  to  the  knowledge-based  assistant  system  CAMA 
{Crew  Assistant  Military  Aircraft)  which  is  presented. 
The  realization  of  CAMA  and  its  functional  units  -  the 
modules  ~  are  described. 

2.  Introduction 

In  future  combat  transport  aircraft,  constraints  created  by 
low  level  flying  in  a  high  risk  theater,  the  high  rate  of 
change  of  information  and  short  reaction  times  required 
will  produce  physiological  and  cognitive  problems  for 
pilots.  From  the  cognitive  point  of  view,  low  level  flying 
over  rapidly  changing  terrain  elevation  coupled  with 
complex  and  dynamic  tactical  environment  will  result 
primarily  in  difficulties  to  maintain  situation  awareness. 
It  still  seems  impossible  to  ensure  the  pilot’s  situation 
awareness  as  the  dominating  requirement  for  high  level 
mission  performance  and  safety. 

The  central  idea  of  a  crew  assistant  is,  to  ensure  that  the 
crew  will  have  all  necessary  and  useful  information  and 
is  acting  under  normal  load  condition,  according  to 
human-centered  automation  [1].  Design  criteria,  which 
aim  at  a  cooperative  function  distribution  between  man 
and  machine  like  that  of  two  partners  [2]  have  to  be 
established. 

Both  man  and  machine  are  active  in  parallel  by  assessing 
the  situation  and  looking  for  conflict  solutions  at  the 
same  time.  In  contrast  with  current  man-machine 
interaction,  both  assist  each  other  while  heading  for  the 
same  goals.  Consequently  Billings  [1]  demands:  „Each 
element  of  the  system  must  have  knowledge  of  the 


others’  intent.  Cross  monitoring  (of  machine  by  human, 
of  human  by  machine  and  ultimately  of  human  by 
human)  can  only  be  effective  if  the  agent  monitoring 
understands  what  the  monitored  agent  is  trying  to 
accomplish,  and  in  some  cases,  why.“  Hence,  the  level  of 
understanding  what  each  element  of  the  system  is  doing 
should  be  as  high  as  possible. 

Therefore,  Onken  [3,  4]  demands,  that  knowledge-based 
aiding  systems  should  comply  with  two  basic 
requirements: 

(1)  "Within  the  presentation  of  the  full  picture  of  the 
flight  situation  it  must  be  ensured  that  the  attention 
of  the  cockpit  crew  is  guided  towards  the  objectively 
most  urgent  task  or  sub-task  of  that  situation." 

(2)  "If  basic  requirement  (1)  is  met,  and  if  there  still 
comes  up  a  situation  with  overcharge  of  the  cockpit 
crew  (in  planning  and  execution),  then  this  situation 
has  to  be  transfered  -  by  use  of  technical  means  - 
into  a  situation  which  can  be  handled  by  the  crew  in 
a  normal  manner." 

Basic  requirement  (1)  is  to  ensure  situation  awareness  of 
the  crew.  In  part,  it  can  be  transferred  into  the  functional 
requirement  for  the  assistant  system  as  of  being  capable 
to  assess  the  situation  on  its  own. 

Pilot’s  workload  has  become  a  critical  issue  as  the 
mission  complexity  has  grown.  It  is  particularly  desirable 
to  reduce  the  need  to  compose  the  relevant  information 
from  numerous  separately  displayed  data.  The  ability  of 
the  assistant  system  to  detect  conflicts,  to  initiate  and  to 
carry  out  on  its  own  the  conflict-solving  process  and  to 
recommend  and  explain  this  solution  to  the  pilot,  gives 
the  pilot  sufficient  time  to  cope  with  unanticipated 
events. 

This  appears  to  be  a  situation-dependent,  flexible  and 
cooperative  share  in  situation  assessment  and  conflict 
resolution  between  the  electronic  and  the  human  crew 
member. 

Automation,  like  recommended  in  the  past,  seems  to  be 
very  attractive  but  it  has  to  be  handled  with  care  not  to 
find  the  pilot  out  of  the  loop  of  conducting  the  mission 
and  flying  the  airplane  (check  „automate“  and  come  back 
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Figure  1 :  Functional  Structure  of  a  KAS 


in  „manual“  if  necessary).  Automation  as  demanded  in 
requirement  (2)  without  the  prerequisite  of  requirement 
(1)  changes  the  pilot’s  task  into  automate  management, 
merely  monitoring  automatic  systems.  Increasing 
workload  of  the  crew  should  lead  to  an  anticipating  of 
future  mission  phases  and  to  tasks-execution  from  that 
part  of  the  man-machine  system,  that  covers  the  more 
comprehensive  and  accurate  situation  comprehension  for 
the  actual  task  [4], 

These  design-criteria  and  requirements  lead  to  a 
functional  layout  of  a  knowledge-based  assistant  system 
(KAS),  which  is  illustrated  in  Figure  1. 

For  a  comprehensive  understanding  of  the  situation,  the 
process  of  situation  analysis  starts  with  the  perception  of 
features.  More  abstract  objects  are  derived  from  these 
features  and  assembled  to  an  overall  situation 
description.  This  closely  resembles  the  human  way  of 
situation  analysis.  On  that  bases  the  situation  diagnosis 
process  recognizes  and  predicts  conflicts  from  observable 
indicators,  caused  by  events  in  the  domain  of  either 
aircraft,  pilot  or  environment. 

Alternatives  for  goals,  plans,  tasks  and  actions  are 
generated  including  that  one,  which  represents  the  given 
flightplan,  and  all  are  checked  with  respect  to  potential 
harmful  conflicts.  If  conflicts  are  detected,  only  the 


conflict-free  alternatives  are  passed  to  the  conflict  solver. 
The  conflict  solving  process  is  ranking  theses  alternatives 
and  selects  the  most  favored  alternative  on  the  basis  of 
the  mission  success  criteria. 

Dialogue  management  insures  effective  communication 
with  the  crew.  This,  the  front-end  of  an  assistant  system 
is  to  present  all  necessary  and  useful  information  in  a 
way,  that  is  easy  to  comprehend.  Messages  to  the  cockpit 
crew  should  be  tuned  and  tailored  to  the  current 
situation,  especially  with  respect  to  the  resources  of  the 
crew.  Inputs  to  the  system  should  allow  initialization  of 
services  and  decision  support  without  tedious  or 
distracting  input  actions. 

Knowledge  processing  needs  a  dynamic  object- 
orientated  representation  of  the  situation-describing 
objects.  The  representation  covers  sensor  data  as  well  as 
very  abstract  objects  like  a  whole  flight  plan  or  for 
instance  the  recognized  intent  of  the  crew. 

Other  knowledge  bases  are  needed  to  express  and  enable 
access  to  domain  knowledge  and  to  permit  inferencing. 
Models  about  motives  and  goals,  task  selection, 
execution  knowledge  and  demand  for  resources  as  well 
as  behavior  models  are  important  examples  of  this  kind 
of  knowledge,  executed  by  additional  information  about 
the  crew  member. 

Static  data  bases  for  navigation  purposes  or  threat  data 
bases  can  already  be  considered  as  standard. 

The  expert  knowledge  embodied  in  the  system  has  to  be 
obtained  in  a  systematic  way. 

Knowledge  acquisition  appears  as  the  bottle  neck  during 
development  of  the  knowledge-based  assistant  system. 
Well-defined  and  efficient  algorithms  and  methods  have 
to  be  used  to  cope  with  the  ill-structured  and  uncertain 
real  world. 

In  order  to  increase  user  acceptance,  it  is  desirable  that 
the  system  contains  a  justification  or  explanation 
component.  First  of  all  the  user  should  be  conscious  of 
the  rules  that  are  applied  in  the  algorithm  to  obtain  a 
solution  or  system  state  to  gain  confidence  to  the  system. 
System  self-diagnosis  makes  sure  that  the  hints  and 
services  to  the  crew  will  be  really  useful.  The  system 
must  be  able  to  realize,  if  information  concerning  the 
actual  situation  might  be  insufficient  to  assist  the  crew, 
or  that  the  system  itself  is  not  working  all  right  and  needs 
to  be  corrected. 

3-  CAMA  -  Crew  Assistant  Military  Aircraft 
3.1  Overview 

CAMA  (Crew  Assistant  Military  Aircraft)  is  a 
knowledge  based  cockpit-assistant-system  for  military 
transport  aircraft.  It  is  being  developed  and  tested  in 
close  cooperation  between  the  DASA  (Daimler-Benz 
Aerospace  AG),  DLR  (German  Aerospace  Research 
Establishment),  ESG  (Elektronik-  und  Logistiksysteme 
GmbH)  and  the  University  of  the  German  Armed  Forces, 
Munich.  The  main  challenge  of  the  tactical  transport 
mission  are  the  complexity  and  dynamic  of  the  tactical 
situation,  the  problems  yielded  by  the  necessity  for  a  low 
altitude  flight  in  compliance  with  a  minimal  risk  routing. 
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Figure  2.  Structure  of  the  Crew  Assistant  Military  Aircraft  (CAM A) 


terrain  collision  avoidance,  etc.,  management  of  fuel  and 
time  constraints,  e.g.  TOT  (time  over  target),  and  the 
missing  or  insufficient  establishments  of  approach  aids. 
CAMA  is  designed  under  regard  to  the  requirements, 
introduced  in  chapter  1.  The  realization  of  the  crew 
assistant  will  be  described  in  the  following  chapters. 

3.2  System  Architecture 

The  system  is  divided  in  several  functional  units,  the 
modules  (Figure  2).  Each  module  is  responsible  for 
specific  tasks.  Communication  between  them  is  realized 
via  the  centralized  situation  representation  (CSR).  The 
CSR  manages  and  provides  static  data  and  situational 
information,  processed  by  the  modules. 

3.3  Situation  Interpretation 

33.1  Environment  Interpreter 

The  environment  interpreter  (El)  evaluates  the  situation 
concerning  weather,  navigational  aids,  air-traffic  and 
flying  objects  (e.g.  SAM).  When  certain  weather  events 
like  thunderstorms  or  clear  air  turbulences  have  occured, 
it  is  used  as  data  source  for  the  module  flight  situation 
and  threat  interpreter  and  the  module  mission  planner. 


3.3.2  Terrain  Interpreter 

Basing  upon  the  present  position  and  course,  the  terrain 
interpreter  (TI)  predicts  the  trajectory  of  the  aircraft, 
generates  a  hyperplane  of  possible  flight  trajectories 
achievable  by  full  exploitation  of  the  aircraft's 
performance  capabilities.  Evaluation  against  a  digital 


terrain  elevation  map  -  basing  on  digital  terrain  elevation 
data  (DTED)  -  detects  potential  terrain  conflicts  in  front 
of  the  aircraft.  A  recommendation  for  an  effective  evasis 
manoeuver  will  be  given. 

3.33  System  Interpreter 

The  system  interpreter  (SI)  [5,  6]  monitors  the  aircraft's 
subsystems  (main  system,  primary  flight  system, 
navigation  systems,  etc.).  Any  detected  malfunction  is 
evaluated  by  an  on  board  diagnostic  to  determine  it’s 
reason. 

33.4  Computer  Vision  External 

The  module  computer  vision  external  (CVE)  [7]  serves 
for  two  purposes.  The  primary  job  is  to  support  a 
ILS/MLS  like  approach  even  on  unequippted  airfields. 
Additionally  it  detects  conflicts  concerning  obstacles  on 
the  runway.  Multiple  sensor  data,  like  gyros, 
accelerators,  GPS,  etc.  are  used  for  aircraft  state 
estimation.  Additionally  a  camera-system  is  used  to 
determine  the  relative  position  to  the  runway. 

3.3.5  Computer  Vision  Internal 

The  module  computer  vision  internal  (CVI)  provides 
information  concerning  pilot's  point  of  gaze.  These  are 
evaluated  by  a  camera  in  the  cockpit's  front-panel,  to  the 
pilot's  opposite.  The  information,  for  instance  the  moving 
line  of  sight  to  a  control  surface  or  to  a  special  indicator, 
could  be  used  to  confirm  the  need  for  a  warning  or  a  hint. 
Especially  with  regard  to  an  additional  resource  model 
that  will  make  modeling  the  crew  more  complete, 
information  of  eye  fixations  is  essential. 
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3.3.6  Tactical  Situation  Interpreter 
The  tactical  situation  interpreter's  (TSI)  main 
contribution  is  the  computation  of  a  threat  map  [8].  The 
calculation  is  based  upon  digital  terrain  elevation  data 
(DIED)  and  the  threat’s  models.  Particular  objects  from 
a  given  list  of  tactical  elements  are  regarded  as  threats 
such  as  surface-to-air  missiles  (SAM)  or  radar  sites.  A 
threat  model  contains  the  parameters 

•  TYiaviniiim  range, 

•  operationabiUty, 

•  efficiency  along  range  and 

•  respective  models  for  threat  area  overlapping. 

Figure  3  shows  the  principle  steps  of  the  algorithm  for 
the  threat  value  calculation. 

Due  to  the  characteristics  of  the  threat’s  radar  systems 
and  respective  radar  shadows  resulting  from  the  terrain 
structure,  the  altitude  above  ground  up  to  which  an 
aircraft  is  not  detectable  by  the  hostile  radar  beams  can 
be  derived  from  the  DTED  database.  Given  a  certain  test 
altitude  a  threat  value  of  zero  can  be  assumed  below 


actions  are  determined,  considering  flight  plan,  local 
ATC  instructions,  aircraft  and  environmental  constraints. 

Normative  Pilot  Behavior 

The  normative  model  [9]  describes  deterministic  pilot 
behavior  as  documented  in  pilot  handbooks  and  air 
traffic  regulations.  Modeling  is  done  primarily  within  the 
domain  of  rule-based  behavior,  but  covers  admissible 
tolerances  also.  Pilot  behavior  can  be  separated  into 
situation  assessment  and  action  processing  components. 
Modeling  is  done  for  all  flight  segments  (taxi,  takeoff, 
departure,  IFR-cruise,  tactical  flight,  drop,  approach, 
landing)  and  concerns  the  following  tasks: 
a)  situation  assessment 
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Figure  4:  Pilot  Behavior  Interpretation  and  Flight  Situation  and  Threat  Interpretation 


these  radar  beams.  Above  the  threat  value  is  calculated  as 
a  function  of  the  individual  model  parameter.  The  threat 
values  are  calculated  for  ten  discrete  altitudes  above 
ground  level  (test  altimdes  every  50  meters  in  the  z-axis) 
and  for  each  terrain  elevation  grid  point 
(longitude/latitude  coordinates).  Area  overlapping  of 
threats  is  taken  into  consideration  by  probability 
calculus. 


3.4  Pilot  Behavior  Interpretation 

3.4.1  Piloting  Expert 

The  piloting  expert  (PE)  is  constructed  as  a  model  of 
pilot  crew  and  covers  normative  and  individual  crew 
behavior.  On  the  basis  of  this  knowledge,  expected  pilot 


•  recognition  of  actual  flight  segment 

•  recognition  of  process  of  plan  execution  related  to 
flight  plan  procedures 

b)  pilot  actions  /  performance 

•  primary  flight  guidance  (altitude,  covuse,  airspeed, 
power  setting,  climb/descent  rate,  pitch  attitude) 

•  drop  procedure 

•  operation  of  flaps,  gear  speed  brakes 

•  opeartion  of  ramp 

•  radio  navigation 

•  communication  with  ATC  and  C&C 

Petri  nets  were  chosen  as  most  suitable  for  knowledge 
representation  purposes. 

Individual  Pilot  Behavior 

The  normative  model  is  being  enhanced  by  providing 
infonnation  on  the  individual  parameters.  Fundamental 
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for  an  adaptive  model  [10]  is  the  assumption  that 
normative  regulations  and  procedures  are  still  the 
guidelines,  even  when  they  are  amended  and  adapted  by 
the  individual  pilot. 

Normative  statements  derived  from  the  petri  net  model 
are  customized  in  order  to  achieve  a  description  of 
individual  behavior  by 

1.  varying  the  transition  parameters  of  the  petri  net  (on¬ 
line) 

2.  varying  the  structure  within  a  petri  net  (off-line) 

3.4.2  Pilot  Intent  and  Error  Recognition 
The  module  pilot  intent  and  error  recognition  (PIER) 
monitors  pilot's  activities  and  mission  events  in  order  to 
interpret  and  understand  the  pilot's  actions  [11]. 
Expected  crew  actions  are  compared  with  the  actual 
behavior  shovm  by  the  crew  (Figure  4).  If  discrepancies 
will  be  detected  the  module  PIER  tries  to  figure  out, 
weather  the  deviation  was  caused  erroneously  or 
intentionally.  Detected  errors  are  issued  to  flight 
situation  and  threat  interpreter.  Error  detection  will  help 
the  pilot  to  correct  slips  and  to  optimize  his  situation 
awareness  during  committing  a  mistake  by  focusing  his 
attention  on  most  important  or  critical  events. 

By  monitoring  pilot  actions  as  well  as  the  mission 
context,  the  system  is  able  to  compare  the  pilot’s  action 
to  a  set  of  behavior  hypotheses.  In  case  of  an  intentional 
deviation  from  the  flight  plan,  the  module  checks,  if  the 
behavior  fits  to  the  given  set  of  intent  hypotheses. 

These  hypotheses  represent  behavior  patterns  of  pilots, 
for  example,  when  commencing  a  missed  approach  or 
avoid  a  thunderstorm.  With  the  intention  recognized, 
support  like  re-planning  is  initiated,  and  the  overall  loop 
could  be  closed  without  further  inputs  of  the  pilot. 


3.5  Flight  Situation  and  Threat  Interpretation 

The  module  flight  situation  and  threat  interpreter  (FTI) 
represents  the  central  module  of  the  assistant.  It  deals 
with  the  situation  assessment  and  conflict  resolution.  The 
process  of  situation  diagnostic  and  the  decision  flnding 
of  how  to  proceed  -  recall  Figure  1  -  are  the  primarily' 
jobs  of  the  FTI.  Situational  objects,  provided  by 
preceding  modules  (e.g.  El,  TI,  SI,  etc.),  are  further 
processed.  The  complete  image  of  the  situation  is 
evaluated  against 

•  goals, 

•  plans  and 

•  pilot  activities. 

Mission  dependent  goals  are  derived  from  the  mission 
order.  The  mission  order  comprises  instructions  and 
constraints,  which  are  to  be  kept  (e.g.  entrance-corridors 
to  gaming  area,  drop-point,  TOT,  etc.).  Taking  into 
consideration  the  mission  order,  the  FTI  initiates  the 
module  mission  planner  (Figure  4)  to  generate  a 
complete,  conflict-free  flightplan.  If  the  mission  order 
leads  the  aircraft  into  a  threatened  area,  the  low  altitude 
flight  planner  is  started  additionally  for  the  calculation  of 
a  low  altitude  flightplan  as  well  as  a  trajectory.  FTI 
controls  the  planning  parameters.  They  are  provided  to 


the  planning  modules  and  comprise  origin  and 
destination,  corridors  to  be  planned  through,  civil  and 
tactical  waypoints  as  well  as  detailed  drop  procedures. 
Crew  constraints,  e.g.  personal  route  preferences  are 
included  as  well.  Generated  routes  are  proposed  to  the 
cockpit  crew,  and  are  accepted,  modified  or  refused 
respectively. 

Misleading  crew  intents  and  errors,  recognized  by  the 
module  PIER  are  monitored  and  crew  warnings  will  be 
initiated.  Intentionally,  conflict-free  deviations  are 
incorporated  in  the  actual  flightplan. 

Monitoring  of  the  flightplan  and  evaluation  against  the 
situation  is  done  permanently.  Conflicts  within  the 
planned  route,  e.g.  new  threats  within  the  operation  area 
or  corridors,  weather  deterioration  en-route  or  at 
destination,  etc.,  will  be  recognized  and  suitable 
resolutions  are  started. 

3.6  Mission  Planning 

3.6.1  Mission  Planner 

The  module  mission  planner  creates  and  maintains  a 
'take  off -to-landing  mission  flight  plan,  including  routes, 
profiles,  time-  and  fuel-planning  based  on  knowledge 
about  the  mission  plan,  gaming  area,  destination,  ATC 
instruction,  aircraft  status,  environmental  data,  etc.  [12]. 
External  data  sources  (C^,  weather,  results  from 
reconnaissance,  etc.)  are  incorporated  into  the  plan. 
Events  like  failures  of  aircraft  systems  (navigational 
equipment,  engines,  etc.)  weather  changes  and  ATC  or 
C&C  instruction  are  taken  into  consideration.  The 
mission  planer  covers  the  flight  under  instrument  flight 
rules  (IFR)  as  well  as  time  management.  Especially  with 
regard  to  a  TOT  (time  over  target),  fuel  calculations  and 
routes/profiles  calculations  the  mission  planner  will 
assist  the  crew.  The  calculated  route  is  presented  as 
proposal  to  be  accepted  or  modified.  The  4D  trajectory 
serves  as  knowledge  source  for  other  function  blocks. 

3.6.2  Low  Altitude  Flight  Planner 

The  aim  of  the  module  low-altitude  flight  planner  (LAP) 
[13]  is  the  calculation  of  a  3D  route  between  the  given 
planning  constraints  —  controlled  by  the  FTI  —  with  a 
maximum  probability  of  survival  in  a  hostile 
environment.  This  is  achieved  by  avoiding  threatened 
areas  if  possible,  minimizing  the  exposure  to  unknown 
threats  and  keeping  clear  of  the  terrain.  Therefore,  the 
planning  constraints,  the  tactical  elements  and  the 
resulting  threat  map,  the  terrain  elevation  data  and  the 
aircraft  performance  data  arc  taken  into  consideration. 
The  system  consists  out  of  three  functional  sub-modules: 

Danger  Analysis 

The  danger  analysis  incorporates  the  threat  map 
calculation  as  described  in  chapter  3.3.6.  Additionally, 
the  visibility  at  each  point  is  calculated  without  assuming 
any  particular  threats.  The  algorithm  issues  lower  danger 
values  on  the  side  of  valleys  than  in  the  center.  Finally, 
the  danger  analysis  utilizes  the  calculation  of  a  ground 
collision  probability,  which  is  particularly  high  in  rough 
terrain.  This  feature  leads  to  generally  higher  flight 
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altitudes  in  the  absence  of  threats.  An  overall  danger 
value  is  calculated  for  each  terrain  grid  point. 

Moding  and  Control 

The  moding  and  control  checks  the  flight  status  and 
assembles  the  target  point  and  the  planning  area  for  the 
optimization  according  to  the  mission  constraints.  The 
optimization  provides  an  array  of  optimal  directions  to 
the  target  point.  As  long  as  a  re-planning  does  not  imply 
a  new  target  point  another  optimization  run  is  not 
required. 

Path  Selection 

The  path  selection  depends  on  the  current  planning  mode 
(initial  planning  or  re-planning).  It  constructs  a  terrain 
grid  based  flight  path  from  a  given  start  point, 
respectively  present  aircraft  position  to  the  target  point. 
The  output  assembly  functions  trajectory  synthesis  and 
plan  analysis  form  the  low -altitude  flight  planner  output. 
In  order  to  be  monitored  by  a  pilot  model  based  assistant 
system,  the  representation  of  the  detailed  trajectory  has  to 
be  reduced  to  a  waypoint  based  low  altitude  flight  plan, 
which  represents  the  general  considerations  to  be 
followed  in  the  human  planning  of  low  level  missions. 
Additionally,  the  low-level  flight  plan  is  given  by  a 
detailed  trajectory  representation. 

3.7  Crew-Interface 

Communication  between  CAMA  and  crew  plays  an 
important  role.  The  kind  of  information  to  be  transmitted 
in  either  direction  varies  with  respect  to  the  different 
modules.  The  information  flow  from  the  machine  to  the 
crew  and  vice  versa  is  controlled  exclusively  by  the 
module  dialogue  manager  (DM)  [14].  The  many 
different  kinds  of  messages  require  a  processing  in  order 
to  use  an  appropriate  display  device  and  to  present  the 
message  at  the  right  time.  The  module  dialogue  manager 
ranks  the  output  messages  and  the  most  important 
message  is  issued  first.  As  output  devices  both,  a 
graphic/alphanumeric  color  display  and  a  speech 
synthesizer  are  used. 

More  complex  information,  e.g.  the  current  flight  plan, 
are  depicted  on  a  horizontal  situation  display.  The 
horizontal  situation  display  is  an  interactive  touch- 
sensitive  map  display  organized  in  a  number  of  layers 
which  allows  the  crew  to  select  optional  map- 
presentations  in  any  combination.  It  allows  to  depict 
tactical  and  threat  information  as  well  as  a  variety  of 
navigational  elements  and  a  topographical  map  similar  to 
the  currently  used  low  flying  chart  paper-maps.  A  second 
display  contains  the  alpha-numeric  flight-log  and  is  used 
for  in-flight  departure-,  approach-  or  missed-approach- 
briefings. 

Commencing  the  approach  to  the  destination  airport,  the 
pilot  will  be  assisted  with  a  combined  linguistic  and 
graphical  briefing,  describing  the  characteristics  and  any 
dangers  associated  with  the  approach. 

Brief  warnings  and  hints  are  used  to  make  the  crew 
aware  of  a  necessary  and  expected  action  and  are 
transmitted  verbally  using  the  speech  synthesizer  [15]. 
The  input  information  flow  is  established  by  use  of 
speech  recognition  in  addition  to  conventional  input 


mechanisms.  Intuitive  direct  voice  input  relieves  the  pilot 
of  a  lengthy  and  tedious  alpha-numerical  input  task. 
Voice  control  seems  to  be  the  best  solution  to  deal  with 
mass  data.  The  total  on-board  vocabulary  will  be  very 
large  and  is  broken  down  into  sub-sets  according  to 
context.  In  order  to  improve  speech  recognition 
performance,  almost  the  complete  knowledge  of  CAMA 
is  used  for  contextual  decoding  to  provide  situation 
dependent  syntaxes.  Thus,  the  complexity  of  the  overall 
language  model  is  reduced  significantly  such  that  the 
system  can  achieve  high  recognition  rates. 

The  use  of  speech  input  and  output  devices  also  reflect 
the  idea  of  human-centered  development  with  respect  to 
effective  communication. 

4.  Conclusion 

The  knowledge-based  system  CAMA  improves  the 
crew's  situation  awareness.  Comprehensive  situation 
assessment  by  the  machine  in  parallel  to  the  crew's 
situation  assessment  is  realized  with  subject  to  a  human- 
centered  automation.  Monitoring,  planning  and  decision 
aiding  functions  provide  a  safe  and  successful  mission 
and  improve  the  mission  effectioness. 

The  actual  integration  phase  of  CAMA  will  end  in  June 
1998  with  a  man-in-the-loop  full  mission  simulation 
campaign.  After  these  simulator  tests  CAMA  will  be 
integrated  in  the  experimental  cockpit  of  the  ATTAS  test 
aircraft  of  the  German  Aerospace  Research 
Establishment  (DLR)  and  will  be  demonstrated  in  flight 
experiments  which  are  scheduled  for  early  1999. 
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1.  SUMMARY 

Automation  of  human  functions  in  complex  dynamic 
systems  produces  problems  for  awareness,  dependency, 
intervention  and  decision-making.  Aiding  and  support 
strategies  for  technological  assistance  to  human  decision¬ 
making,  which  embody  notions  of  Human  Electronic-Crew 
(HEC)  teamwork,  are  proffered  as  alternatives  to 
automation.  This  paper  considers  the  relevance  of  control 
theory  as  a  broad  framework  for  HEC  teamwork  in  dealing 
with  uncertainty  in  dynamic  situations.  It  examines  possible 
“cognitive  cockpit”  architectures  for  future  HEC  systems, 
which  adapt  to  both  control  feed-back  and  feed-forward 
requirements,  encompassing  the  uncertainy  in  dynamic 
situations. 

2.  INTRODUCTION 

Recent  developments  in  real-time  data  acquisition,  fusion, 
and  processing,  and  in  computer  modelling  and  AI 
modelling  techniques  (Expert  Systems,  KBS,  Neural 
Networks),  are  being  used  to  provide  aiding  and  support 
for  aircrew  tasks  e.g.  Co-pilote  Electronique,  CASSY 
Cockpit  Assistant,  Pilot’s  Associate  technologies.  These 
Human  Electronic  Crew  technologies  aim  to  enable  better 
management  of  aircrew  workload,  and  to  enhance  situation 
awareness  (SA).  Ambitiously,  they  aim  to  improve  the 
quality  of  decision-making  in  uncertain  situations  by 
involving  both  the  human  and  computer  system 
components  in  the  performance  of  cognitive  tasks  and 
functions  i.e.  joint  cognitive  HEC  systems.  The  potential 
for  co-operation  is  achieved  by  incorporating  a  model  of 
human  decision  making  and  control  abilities  into  the 
control  automation,  by  monitoring  pilot  performance  and 
workload  through  behavioural  and  physiological  indices, 
and  by  predicting  pilot  expectations  and  intentions  with 
reference  to  embedded  knowledge  of  mission  plans  and 
goals. 

These  ideas  have  led  to  the  need  for  greater  scmtiny  of  the 
specification  of  cognitive  requirements,  candidate  cognitive 
architectures,  and  appropriate  cognitive  engineering 
principles  for  such  joint  HEC  cognitive  systems. 
Paramount  among  these  concerns  are  the  particular 
requirements  for  control,  dynamic  function  allocation 
(DFA),  and  computer  autonomy  in  such  complex  HEC 
systems,  with  important  implications  for  human  command, 
authority,  tmst,  feedback,  and  error  rectification. 

One  area  of  practical  work  on  the  control  issues  has  been  to 
seek  interface  solutions  through  pilot  selectable  levels  of 
autonomy  (LOA)  for  functions,  governed  by  the  required 
pilot  operational  relationship  and  interaction.^^^  Discrete 
LOA  modes  have  been  proposed  (Inactive,  Standby, 
Advisor,  Assistant,  Associate),  based  on  Sheridan’s 
taxonomy,  with  tailorable  functional  clusterings  for  flexible 
responding,  to  avoid  too  rigid  automation  imposed  by 
design.  In  Associate  mode,  under  full  DFA,  the  proposed 


system  maintains  advisory  functions  and  accepts  pilot 
allocated  tasks,  but  also  takes  over  tasks  as  the  context 
demands.  These  modes  aim  to  provide  bounded, 
communicable  structure  for  delegated  levels  of  authority, 
minimising  mode  confusion,  and  building  trust  and 
confidence.  But  the  required  control  structure  should  not  be 
cognitively  complex.  Pilots  tend  to  view  EC  autonomy 
simply  as  either  automatic,  with  or  without  status  feedback; 
semi-automatic,  describing  what  will  happen  and  asking 
permission  to  proceed;  or  advisory,  providing  information 
only.^*^^ 

3  AUTOMATION  AIDING  EXPERIMENTS 

3.1  Co-operation  and  Compensation 

Control  problems  arising  from  poor  mode  awareness  are 
commonly  reported  with  complex  flight-deck  automation 
systems.  In  recognition,  experimental  work  has  examined 
the  logic  or  rules  for  automatic  invocation  of  levels  of 
aiding,  to  maintain  pilot  skill  and  avoid  complacency  e.g. 
cyclical  invocation. 

Following  this  research  paradigm,  an  investigation  of 
awareness  of  levels  of  flight  automation  aiding  was  varied 
was  reported  at  the  3rd  HEC  workshop  Using  the 
NASA  Multi  Attribute  Task  (MAT)  adaptive  automation 
simulation  software,  failures  were  introduced  in  the  ability 
of  the  automation  to  intervene  in  a  timely  and  appropriate 
manner,  with  regard  to  the  prevailing  task  demands.  The 
invocation  logic  was  manipulated  to  be  either  logical  (co¬ 
operative,  reducing  workload),  simulating  normal  aiding,  or 
illogical  (uncooperative,  increasing  workload),  simulating 
automation  failure.  The  results  showed  that  manual 
compensation  generally  equated  performance  across  the 
conditions  but  without  awareness  of  the  automation  failure, 
indicated  by  subjective  measures.  Monitoring  and  tracking 
of  the  status  and  functioning  of  aiding  with  automatic 
invocation  is  difficult  to  achieve.  Design  safeguards  would 
be  needed  to  prevent  unacceptable  unpredictability  with 
variable  function  allocation,  and  to  minimise  inappropriate 
allocation  e.g.  establishing  operator  willingness  to  accept 
aiding.  Recent  research  at  the  University  of  Minnesota  has 
shown  how  an  acceptable  invocation  logic  can  be  based  on 
significant  critical  events  in  the  mission  situation,  e,g. 
avoiding  obvious  hazards  i.e.  mission  context  sensitive. 

3.1  Control  Feedback  Awareness 

The  MAT  adaptive  automation  simulation  software, 
comprises  three  task  elements,  namely  manual 
Compensatory  Tracking  (CT),  Systems  Monitoring  (SM), 
and  Resources  Management  (RM),  SM  and  RM  tasks  were 
operated  in  manual,  semi-automatic,  or  fully  automated 
modes.  Task  aiding  is  invoced  automatically,  with  on¬ 
screen  caption  warning  and  feedback  on  mode  change  and 
status.  The  basic  MAT  battery  provided  no  explicit 
differentiation  between  manual,  automatic  and 
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environmentally  induced  screen  changes.  Automation 
functioning  has  to  be  inferred  from  understanding  of  the 
control  rules. 

To  investigate  more  intuitive,  cognitively  compatible  (CC) 
iconic  displays  of  automation  status  a  followup  study,  with 
a  modified  version  of  the  MAT  battery  display  screen  was 
created^^^  (Figure  1).  It  was  considered  that  a  familiar, 
dynamic  physical  instantiation  of  automation  functioning, 
using  icons  with  properties  of  agency,  might  assist  in 
understanding  automation  status.  In  the  modified  MAT 
battery,  automation  activity  was  represented  using  dynamic 
visual  icons,  depicting  the  form  and  location  of  control 
actions.  Droid-like,  R2D2  characters  emerged  when  the 
automation  was  invoked.  These  Adaptive  Iconics  (AIs) 
were  re-positioned  and  animated  beside  SM  parameters  and 
RM  functional  elements,  where  the  automatic  action  was 
perceived  as  taking  place.  An  experiment  with  20  non¬ 
aircrew  compared  performance  with  the  original  and  AIs- 
modified  MAT  task.  Subjects  flew  task  profiles  with 
increasing  frequency  of  events  requiring  actions,  with 
logical  and  illogical  aiding  invocation.  Ratings  of  CC  using 
the  SRK-related  experiential  CC-SART  measure 
(Dimensions:  Level  of  processing;  ease  of  reasoning; 
activation  of  knowledge),  showed  enhanced  CC  by  AIs  of 
the  RM  task  (and  CT),  with  improved  RM  ease  of 
reasoning.  However,  SM  performance  was  poorer  with  the 
AIs.  This  was  interpreted  as  over-reliance  on  SM 
automation  with  partial  aiding,  induced  by  the  presence  of 
AIs,  without  clear  indication  of  their  restricted  support-  AIs 
improved  the  cognitive  quality  of  the  interface  for  one  rule- 
based  task,  but  created  undesirable  side  effects  on 
performance  and  goal  tracking  for  another. 

3.2  Microworld  Uncertainty 

The  MAT  environment  provides  a  “nodcroworld”  of  flight 
simulation,  where  the  task  situations  were  both  complex, 
dynamic,  and  opaque:  not  everything  is  visible,  and 
required  inferences  with  uncertainty^^l  To  examine  the 
structure  of  the  MAT  microworld,  a  ‘goals-means’ 
cognitive  task  analysis  (CTA)  was  performed  which 
classified  the  individual  MAT  tasks  according  to  which 
knowledge  structures  were  being  activated,  which  rules 
were  being  applied,  and  what  skills  were  being  drawn  upon 
to  perform  the  tasks.  This  CTA  revealed  that  performance 
on  the  MAT  CT  task  was  skill-based,  that  the  MAT  SM 
and  RM  tasks  were  both  rule-based,  and  that  some 
knowledge-based  performance  was  possible  in  the  RM 
task.  However,  the  support  provided  by  the  automation  was 
entirely  rule-based.  Thus,  the  MAT  environment  provides 
only  a  weak  microworld  test  of  automation  aiding, 
restricted  to  rule-based  problems.  Accordingly,  in  order  to 
provide  a  broader  microworld  basis  for  assessing 
automation  aiding,  we  have  developed  a  modified  VAPs  C- 
code  version  of  the  ATC  Simulation  Task,  used  previously 
in  SA  metrics  research,  adapted  to  simulate  a  multi-aircraft 
military  mission  (Figure  2).  This  provides  some  of  the 
missing  navigation,  guidance,  hazard/threat  management, 
and  spatial  reasoning  task  components  of  real  world  of 
military  aviation,  which  is  a  rich  source  of  uncertainty 
requiring  knowledge-based  behaviour.  We  believe  that 


uncertainty  is  a  critical  feature  of  microworlds  for  testing 
adaptive  automated  decision  aiding.  In  the  real  world,  for 
at  least  the  forseeable  future,  EC  can  not  know  everything. 
EC  will  be  fallible,  no  matter  how  AI  ‘smart’.  EC  will  need 
human  oversight,  checking  and  directing.  This  is  because 
the  human  crewmember  will  be  able  to  bring  knowledge 
that  EC  does  not  possess,  such  as  high  level,  mobile 
political  and  military  goals,  associated  rules  of  engagement, 
tolerance  of  risk,  or  whatever  uncertainties  human 
intentionality  brings  to  the  situation. 

3.3  Prepared  Flexibility 

The  developed  military  mission  microworld  is  particularly 
suitable  for  testing  mission  planning  and  automation 
concepts.  Automation  of  mission  planning  functions 
anticipates  predictable  events  and  brings  forward  decisions 
on  the  mission  time  line.  In  theory,  plan  automation 
enhances  SA  by  freeing  limited  attentional  resources  for 
unexpected  events.  We  have  sought  to  measure  how 
cognitive  rigidity  or  ‘set  arising  from  plan  automation’ 
inhibits  creativity  and  responsiveness  to  changed  situations 
with  knowledge-based  tasks.  In  an  experiment  simulating  a 
multi-aircraft  ground  attack  mission,  subjects  (30  non¬ 
aircrew)  were  required  to  control  aircraft  over  a  target,  and 
to  exit  the  area  safely,  avoiding  radar  detection  and  enemy 
aircraft  contact.^’^  Survival  was  the  briefed  prime  directive. 
Subjects  received  different  levels  of  mission  support,  i.e.  ad 
hoc  preparation;  automatic  planning  and  mission  rehearsal; 
auto-plan,  rehearsal,  plus  automatic  plan  execution.  Manual 
override  of  automatic  plan  execution  was  provided,  if  the 
subject  wished  to  depart  from  the  plan.  Unexpected  enemy 
aircraft  eventually  appeared  presenting  a  direct  threat  to  all 
aircraft  on  the  planned  heading  to  the  exit  gate,  thus 
requiring  manual  intervention.  The  results  showed 
advantages  for  automatic  plan  generation  and  execution 
with  no  threat  aircraft  present.  However,  when  the  planned 
route  came  under  attack,  rather  than  freeing  attentional 
resources,  the  automation  produced  over-dependence  or 
“blind  trust”  in  the  plan,  resulting  in  high  losses  from 
enemy  contact  (Figure  3,  4).  Subjective  ratings  (SART, 
CC-SART),  indicated  poor  threat  awareness  with  plan 
automation.  Subjects  exhibited  delusion  of  control,  rigidly 
interpreting  experience  through  cognitive  schema  set  by  the 
plan.  The  expected  lag  in  recognition  of  the  changed 
situation,  normally  ascribed  to  schema  refinement,  appears 
exaggerated  by  plans,  causing  extended  awareness 
hysteresis.  Active  involvement  in  plan  generation  and 
execution,  provides  better  adherence  to  directives, 
enhanced  goal  awareness  and  better  strategic  control.  Plan 
automation  creates  a  form  of  goal  blindness:  failure  to  see 
the  goal  for  the  plan.  Therefore,  immersive  planning  and 
rehearsal  technologies,  and  assertive  automation,  should  be 
implemented  with  caution.  Mission  support  should  help  to 
prototype  and  critique  responses  to  unexpected  events.  A 
more  useful  focus  can  be  on  aiding  decision-making  when 
the  plan  breaks  down,  and  on  supporting  reactive  re¬ 
planning.  This  should  be  done  in  ways  that  explain  and 
critique  plan  options  rather  than  that  provide  a  potentially 
rigid  mind  set.  Essentially,  what  is  needed  is  decision 
support  that  can  break  an  inappropriate  mind  set,  without 
substituting  another  i.e.  keeping  the  pilot  in  control. 
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4.  COGNITIVE  CONTROL 

Team  function  allocation  may  be  appropriate  for 
controlling  systems  with  discrete,  bounded,  naturally 
separable  functions  and  tasks,  where  any  resultant 
autonomy  does  not  significantly  threaten  goal  maintenance. 
With  ill-structured  problems  involving  uncertainty, 
function  allocation  needs  to  be  flexible  and  dynamic  with 
good  communication  between  team-members.  Mature 
human  teamwork  involves  good  communication,  function 
and  leadership  initiative  tum-taking,  with  a  transitioning  of 
authority  that  is  smooth  and  flexible.  The  locus  of  control 
is  driven  more  by  situation  and  context,  than  by  the 
preservation  of  a  sole  source  of  control  authority.  Aiding 
technologies  should  use  such  frameworks  to  support  the 
pilot  efficiently  and  unobtrusively  in  achieving  the  mission 
objectives,  while  allowing  the  pilot  to  remain  at  the  top  of 
the  system  control  hierarchy,  with  the  ultimate 
responsibility  for  generating  and  setting  goals  and 
directives  i.e.  staying  in  charge.  Goal  tracking  and  control 
feedback  seem  to  be  the  key  to  success. 

However,  as  tasks  become  more  about  thinking  than  doing 
-  more  cognitive  than  physical  in  nature  -  the  validity  of 
applying  team  function  allocation  has  to  be  questioned. 
Analysis  of  cognition  into  separable  functions,  which 
become  candidates  for  automation,  may  be  counter¬ 
productive  e.g.  mission  planning.  In  joint  cognitive  HEC 
systems,  cognitive  control  may  benefit  from  a  functional 
architecture  with  integration,  rather  than  segregation  and 
allocation,  of  high  level  functions. 

In  considering  cognitive  functional  requirements,  candidate 
cognitive  control  architectures,  and  engineering  principles 
for  HEC  systems,  it  is  our  believe  that  a  principled 
approach  must  be  taken,  based  upon,  guided  by,  and  hence 
traceable  to  theory.  For  this  reason,  we  have  sought  to 
develop  an  approach  to  test  extant  theory  on  the  cognitive 
control  of  complex  systems.  The  ideas  of  Rassmussen 
concerning  the  reduction  of  human  error  in  the  control  of 
complex  systems  have  been  particularly  influential.  This 
provides  an  error-based  classification  of  behaviour  with 
skill,  rule,  and  knowledge  levels  of  performance 
corresponding  to  decreasing  levels  of  familiarity  with  the 
task  or  environment,  or  expertise  (Figure  5).  Thus, 
behaviour  is  driven  by  both  goals  and  by  experience.  This 
approach  recognises  that  humans  are  (and  should  be) 
allowed  to  be  flexible  and  variable,  and  so  error 
observability  and  reversibility  are  important  features  for 
safe  task  and  system  design.  It  is  not  clear  how  control  is 
passed  between  SRK  levels. 

To  account  for  this  and  the  orderliness  of  human  action, 
beyond  a  stored  programme,  Hollnagel  considers  that 
control  of  actions  to  achieve  goals  exists  on  a  continuum  of 
modes,  with  different  planning  horizons,  determined  by 
competence  (c.f.  experience)  and  context,  rather  than 
procedure.  These  modes  range  from  scrambled, 
opportunistic,  to  tactical  and  strategic  cognitive  control 
modes  with  increasing  levels  of  depth  in  the  evaluation  of 
outcomes  with  reference  to  goals. 


Brehmer*^^^  has  argued  that  in  dynamic  systems,  the 
observability  of  the  system  state  and  the  possibilities  for 
action  affecting  the  state  of  the  system  are  key  properties  of 
the  system  to  be  controlled.  The  goals  and  models  of  the 
system  are  properties  of  the  controller  or  decision-maker.  A 
system  may  be  controlled  by  a  feed-forward  or  feedback 
strategy,  or  by  some  combination.  In  feedback  control,  the 
controller  uses  only  current  information  about  the  actual 
state  of  the  system.  In  feed-forward  control,  the  controller 
uses  a  model  of  the  system  to  predict  its  state  and  to  select 
the  appropriate  control  inputs.  Feedback  control  is 
effective  when  there  are  no  significant  feedback  delays  in 
the  system,  when  the  system  changes  over  time  and  no 
stable  model  of  the  system  can  be  constmeted.  Feed¬ 
forward  control  requires  that  the  system  is  stable,  so  that 
the  model  remains  valid.  Applications  of  control  theory  in 
the  automatic  control  of  systems  often  rely  on  the  model  to 
produce  the  actual  control  inputs  and  use  feedback 
information  to  update  the  model.  Feedback  control  is 
cognitively  simpler,  and  is  the  preferred  mode  of  control  in 
dynamic  decision  tasks.  This  works  if  the  rate  of  change  in 
the  controlled  system  is  slow  enough  for  the  feedback 
information  to  be  processed.  Hollnagel  argues  that  if 
there  is  too  much  information  for  the  controller  to  process, 
then  the  response  will  be  delayed  and  the  performance  will 
deteriorate.  Humans  can  use  heuristics  to  gain  time,  but  the 
feedback  becomes  less  precise.  Here,  it  becomes  desirable 
to  rely  more  on  feed-forward  and  to  anticipate  responses.  It 
is  important  that  the  joint  cognitive  system  retains  control 
of  the  process  rather  than  being  controlled  by  it,  and  that 
the  required  stable  equilibrium  is  obtained  by  a  judicious 
blend  of  feed-forward  and  feedback  control. 

In  consideration  of  the  importance  of  supporting  pilot  SA 
through  HEC  systems,  we  have  sought  also  to  adapt 
Perceptual  Control  Theory^^^^  as  a  theoretical  driver  for  this 
area  of  research.  Figure  6  shows  the  Integrated  Model  of 
Perceived  Awareness  ConTrol  (IMPACT).  The  main  tenet 
of  this  particular  model  is  that  SA,  or  the  individual’s 
perception  of  it,  is  controlled  by  behaviour.  The  model 
interprets  behaviour  as  an  attempt  to  minimise  the 
difference  between  desired  and  actual  SA.  It  has  four  main 
constituents:  a  Perceived  Level  of  SA  [P];  a  Desired  Level 
of  SA  [D];  the  User’s  Behaviour  [B];  and  the  Environment 
[E]. 

The  model  works  as  a  basic  closed-loop  feedback  system 
with  a  comparator  incorporating  the  feedback  in  attempting 
to  achieve  the  goal  state.  If  an  individual  has  a  demand  for 
a  higher  level  of  SA,  and  so  increases  [D],  then  when  the 
level  of  [P]  is  found  to  be  insufficient,  the  user  will 
formulate  some  type  of  behaviour  [B],  which  the  user 
believes  will  cause  changes  in  the  environment  [E]  that  will 
decrease  the  difference  between  [P]  and  [D].  For  example, 
if  the  pilot  decides  that  he  wants  to  increase  his  SA,  [D], 
with  relation  to  his  route-plan  then  he  might  select,  and 
check,  the  digital  map  situation  display,  [B],  which  in 
effect,  changes  the  environment  [E].  The  environment  [E] 
then  provides  the  pilot  with  sensory  information,  [s],  that 
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changes  his  perceived  level  of  SA,  [P],  thus  bringing  it 
closer  to  the  desired  level  of  SA,  [D]. 

A  concern  with  the  application  of  the  model  is  how  it  can 
be  used  effectively  by  an  automated  system  to  best  support 
the  pilot.  There  are  two  main  ways  the  automation  can  use 
the  information  the  model  contains.  Firstly,  the  model 
provides  a  reasonable  amount  of  information  on  the  pilot’ s 
sensoiy  input.  By  analysing  what  perceptions  the  pilot  is 
taking  into  account  in  the  decision  process,  and  the  quality 
of  these  decisions  against  closure  on  the  goal,  the 
automation  can  determine  the  pilot’s  informational 
requirements.  Thus,  the  model  will  guide  communication, 
and  meet  the  high  level  requirement  of  ensuring  that  the 
automation  provides  an  appropriate  level  of  information,  of 
an  appropriate  quality.  Secondly,  the  model  provides  some 
explanation  of  the  pilot’s  behaviour,  in  terms  of  goals,  and 
perceptions.  The  ability  to  know  why  an  entity  is 
performing  a  certain  action,  or  series  of  actions,  is  an 
essential  component  of  teamwork.  Similarly,  if  we  apply 
the  model  to  both  the  pilot,  and  the  Electronic 
Crewmember  (Figure  7)  then  an  information  exchange  can 
be  modelled,  the  automation  receives  information  about  the 
pilot,  while  it  also  provides  the  pilot  with  information 
about  the  automation,  and  it’s  actions.  A  similar  symbiotic 
structure  could  apply  to  human-human  team  SA. 

Overall,  the  main  benefit  of  this  model  is  to  guide 
communication,  and  provide  information  about  both  pilot 
and  Electronic  Crewmember.  This  leads  to  an  increased 
awareness,  which  would  benefit  many  complex  decision 
making  processes,  especially  rapid,  reactive  re-planning. 

5.  THE  COGNITIVE  COCKPIT 

Synthesising  the  lessons  learned,  it  seems  that  goal  control 
is  the  key  to  successful  teamwork.  We  have  sought  to 
develop  an  approach  to  an  intelligent  H-EC  Cognitive 
Cockpit  (Figure  8),  designed  to  be  cognition  sensitive, 
compatible,  adaptive,  and  supportive  to  control  of  pilot 
goals,  in  accordance  with  cognitive  engineering  principles. 
This  is  done  by  structuring  all  automated  support  using 
Rasmussen’s  SRK  framework,  which  ensures  both 
invocation  and  representation  of  the  automation  are 
cognitively  compatible.  Using  a  co-operative  perceptual 
control  model,  we  are  developing  principles  for  supporting 
goal  awareness  (current  &  desired)  and  error  awareness 
(diagnosis  &  rectification),  tailored  to  SRK  requirements, 
with  consideration  of  Hollnagel’s  modes  of  control.  For 
conceptual  prototyping,  in  the  current  version  of  the 
Cognitive  Cockpit,  feedback  and  feed-forward  information 
is  provided  to  the  pilot  through  not  only  AIs,  but  also 
through  schema-based  Goal  Balls,  indicating  action-to-goal 
effectiveness  and  risk,  and  providing  a  cognitively 
compatible  and  ecologically  valid  representation  of 
uncertainty  (Figure  10).  SystemCrew  Balls  represent 
supplied  human  versus  EC  workload  against  the  required 
workload.  Support  assertiveness  is  tailored  for  goal  closure 
in  uncertainty,  using  tutoring,  expert  advisor,  and  critiquing 
techniques,  intended  to  overcome  cognitive  rigidity  without 
substitution  by  EC  mind  set.  This  conceptual  approach 
could  resolve  the  conflicting  control  requirements  for 


teamwork  and  autonomy  with  DFA,  by  developing  a  view 
of  EC  as  an  extension  of  pilot  cognitive  functioning  dealing 
with  uncertainty,  rather  than  as  an  independent  cognitive 
agent.  As  a  guiding  principle,  we  believe  that  the  concepts 
within  a  Cognitive  Cockpit  should  be  designed  to  support 
intentionality.  Intentionality  is  a  description  of  an  internal 
mental,  or  cognitive  state  involving  a  focussing  of  effort 
and  attention  on  the  real  world,  with  a  high  level  of 
situation  awareness,  and  with  a  desire,  plan,  or  purpose  of 
achieving  some  externally  referenced  object,  or  goal.  In 
Perceptual  Control  Theory  terms  it  is  the  cognitive  closure 
between  the  perception  of  the  current  situation  and  the 
desired  situation.  To  support  intentionality,  Cognitive 
Cockpit  work  on  individual  differences  (characterology) 
has  indicated  that  we  should  be  concerned  with  supporting 
both  insight  and  responsivity.  Specifically  it  should 
liberate  intentionality,  and  provide  goal,  error  and  control 
awareness.  It  should  not  block  intentionality,  inhibit  nor 
reverse  intentionality,  nor  second  guess.  The  ultimate  aim 
of  the  Cognitive  Cockpit  work  is  to  find  the  appropriate 
blend  of  feed-forward  and  feedback  information  for 
supporting  the  intentions  of  commanders  and  operators, 
that  enables  them  to  be  in  control  of,  rather  than  controlled 
by,  the  system. 
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Figure  1  -  Modified  Version  of  the  MAT  Battery 
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Figure  4-Subject  record  with  Automated  Plan  Execution 
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of  discriniination  -  failure  to  telecl  appropriate  level  of  belt 
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Figure  5-  Rasmussen’s  Skill,  Rule  and  Knowledge  Framework 
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Perceptual  Control  of  Situational  Awareness 


Figure  6  -  IMPACT 
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Figure  7  -  Joint  IMPACT 
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Figure  9  -  Goal  Balls  Hierarchy 
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Figure  10  -  Health  Monitor  Hierarchy 


(c)  British  Crown  Copyright  1997/MOD 
Published  with  the  permission  of  Her  Britanric  Majesty’s  Stationary  Office 


87 


TT^ 


THIS  PAGE  BSm  ^  fONALLY  LEFT  BLANK 


88 


Machine  Perception  as  Electronic  Crewmember  Capability 

S  Fiirst,  S  Werner,  D  Dickmanns  and  E-D  Dickmanns 
University  of  the  German  Armed  Forces  Munich,  GE 

A  machine  perception  system  for  aircraft  and  helicopters  using  multiple  sensor  data  for  state 
estimation  is  presented.  By  combining  conventional  aircraft  sensors  like  gyros, 
accelerometers,  artificial  horizon,  aerodynamic  measuring  devices  and  GPS  with  vision  data 
taken  by  conventional  CCD-cameras  mounted  on  a  pan  and  tilt  platform,  the  position  of  the 
craft  can  be  determined  as  well  as  the  relative  position  to  runways  and  natural  landmarks. 

The  vision  data  of  natural  landmarks  are  used  to  improve  position  estimates  during 
autonomous  missions.  A  built-in  landmark  management  module  decides  which  landmark 
should  be  focused  on  by  the  vision  system,  depending  on  the  distance  to  the  landmark  and  the 
aspect  conditions.  More  complex  landmarks  like  runways  are  modeled  with  different  levels  of 
detail  that  are  activated  dependent  on  range.  A  supervisor  process  compares  vision  data  and 
GPS  data  to  detect  mis-tracking  of  the  vision  system  e.g.  due  to  poor  visibilitv  and  tries  to 
reinitialize  the  vision  system  or  to  set  focus  on  another  landmark  available.  During  landing 
approach  obstacles  like  trucks  and  airplanes  can  be  detected  on  the  runway. 

The  system  has  been  tested  in  real-time  within  a  haidware-in-the-loop  simulation.  Simulated 
aircraft  measurements  com:5)ted  by  noise  and  other  characteristic  sensor  errors  have  been  fed 
into  the  machine  perception  system;  the  image  processing  module  for  relative  state  estimation 
was  driven  by  computer  generated  imagery.  Results  fiom  real-time  simulation  runs  are  given. 
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Cognitive  Architectures  for  Supporting  Strategic  Behaviours  in  Adaptive  Systems 


Dr.  Karmen  Guevara 
swell  Road 

London,  NW3  ILH,  England 


1.  SUMMARY 

This  paper  discusses  the  design  implications  for  human 
electronic  air  crew  systems  from  the  findings  of  recent 
research  on  the  cognitive  processes  underlying  the  strategic 
behaviours  and  decision  making  of  fighter  pilots.  A  research 
aim  was  to  explore  whether  Psycognition,  a  methodological 
approach  which  focuses  on  eliciting  the  subconscious 
processes  which  influence  human  behaviour,  could  contribute 
to  our  understanding  of  the  cognitive  requirements  for 
adaptive  air  crew  systems. 


The  identification  of  predictive  subconscious  behaviours  at 
breakdown  points  can  contribute  to  our  understanding  of  the 
human  requirements  for  future  cognition  adaptive  systems. 
The  paper  considers  the  design  implications  of  this  research 
for  candidate  cognitive  architectures  for  human  electronic 
air  crew  systems.  The  consequences  for  embedding  this 
knowledge  (Psycognition)  in  system  pilot  models  and  HEC 
interfaces  will  be  discussed.  The  implications  of  real  time 
intervention  through  aiding  and  supporting  strategies  will  be 
explored. 


The  Psycognition  methodology  was  applied  to  an 
investigation  of  how  fighter  pilots'  subconscious  processes 
influenced  their  strategic  behaviours  in  handling  certain 
critical  incidents.  The  research  focused  on  pilot’s  strategic 
behaviours  in  three  kinds  of  critical  incidents:  plan 
breakdown,  control  breakdown  and  information  overload. 

The  key  findings  are: 

•  Despite  the  similarities  in  background,  training,  experience 
and  the  strength  of  the  military  culture,  there  were 
significant  differences  in  how  the  subjects  reacted  to  critical 
incidents. 

•  In  these  situations  the  subjects  drew  upon  deeply  rooted 
subconscious  core  beliefs  to  guide  their  decisions  and 
actions,  instead  of  conscious,  rational  cognition. 

•  The  differences  in  sfi-ategic  behaviours  were  evident  in 
situations  which  involved  a  breakdown  in  plan  or  control,  an 
overload  of  information  or  a  compromise  of  principles  and 
values. 

•  Psycognition  provides  us  with  a  basis  for  predicting  what 
these  behaviours  will  be,  the  strategies  that  will  be  applied 
and  the  breakdown  situations  in  which  they  will  be 
triggered. 

Table  1 :  An  Overview  of 


2.  INTRODUCTION 

Our  recent  research  studied  the  strategic  behaviours  of  fighter 
pilots  in  the  handling  of  three  kinds  of  critical  incidents:  plan 
breakdown,  control  breakdown  and  information  overload.  A 
research  aim  was  to  explore  whether  Psycognition,  a 
methodological  approach  which  focuses  on  eliciting  the 
subconscious  processes  influencing  human  behaviour,  could 
contribute  to  our  understanding  of  the  cognitive  requirements 
for  adaptive  air  crew  systems. 

Core  to  Psycognition  is  characterology,  which  was  the 
framework  used  to  examine  the  subjects’  core  strategies. 
Characterology  refers  to  the  set  of  core  beliefs  and  emotional 
responses  formed  early  in  development  and  the  strategic 
behaviours  which  are  predicated  on  these  core  beliefs.  The 
framework  consists  of  six  types  of  character  strategies.  (Ref  1) 
An  abbreviated  overview  of  the  characterological  types  is 
provided  in  Table  1. 


Characterological  Types 


Orientation 

Core  Belief 

Strategic  Behaviour 

Mr.  Safety* 

Safety  &  trust. 

Dangerous  world. 

Over  focused  on  detail  &  analysis. 

Creates  own  world  -  excludes  external. 
Information  overload:  chunk,  filter,  escape. 

Mr  Action* 

Performance 
&  recognition. 

Self-wonh  stems 
from  achievement. 

Action  &  perfection. 

Focus  on  the  logical  &  rational. 

Information  overload:  speed  up. 

Mr  Endurance* 

Indirect  control 
&  endurance. 

Not  good  enough  but 
must  do  one’s  best. 

Subtle  influence  &  control. 

Bearing  up,  delaying,  resisting. 

Information  overload:  queuing  &  delaying. 

Mr  Freedom* 

Freedom  /  control. 
Be  the  best  /  win. 

Must  be  in  charge. 

Not  safe  to  give  up  control. 

Ensures  own  choices  &  decisions. 

Seeks  adventure  &  excitement. 

Information  overload:  abstraction, 
multiple  channels,  manipulation. 

Mr  Self-Reliant 

Challenge,  going 
it  alone. 

Take  care  of  oneself. 

Never  rely  on  others. 

Proving  self-reliance. 

Mobilising  self-support. 

Personal  challenge. 

Mr.  Attention 

Getting  attention 
&  involvement. 

Not  being  interesting 
&  listened  to. 

Dramatises  events  /  feelings  to  get 
attention  &  avoid  separation. 
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Table  2:  Themes  &  Patterns  in  the  Subjects'  Strategic  Behaviours 


Strategy 

Information  Overload 

Control  Breakdown 

Plan  Breakdown 

A 

Mr. 

Safety 

Not  trusting  information. 
Seeks  quantity  in  order  to 
control  overload. 

Unsafe  not  to  be  in  control. 

Feels  unsafe  when  gives  up 
control. 

Focus  on  manipulating 
situation  to  achieve  goal. 

Refusal  to  give  up. 

Withdraws  goal  if  core 
values  arc  compromised. 

B 

Mr. 

Action 

Speeds  up  -  faster. 

Goes  for  detail. 

Increases  effort  -  tries  harder. 
Gives  up  if  doesn’t  reflect 
negatively  on  him. 

Focus  on  what  is  believed  to  be  right. 
Perseveres  &  changes  strategic  plan. 
Accepts  breakdown  if  rationally 
understood. 

C 

Mr. 

Endurance 

Slows  /  delays  things. 

Does  best  &  waits. 

Attempts  to  understand. 
Relinquishes  control. 

Focus  on  doing  his  best. 

Adapts  to  breakdown. 

Compromises  if  necessary. 

D 

Mr. 

Freedom 

i _ 

Deflects  situation 
through  abstraction. 
Manipulates  to 
maintain  control. 

Exerts  power  to  control. 
Superimposes  own  methods 
for  control. 

Focus  on  achieving  what 
he  wants,  in  his  own  way. 

Impulse  over-rides  rational 
thinking  /  judgement. 

Refuses  to  accept  failed  situation. 

The  research  highlighted  significant  differences  between  the 
subjects’  strategic  behaviours  in  the  three  incidents,  which  led 
to  four  particular  strategies  emerging  from  the  data.  (Ref  2) 
The  strategic  themes  that  emerged  suggested  that  each  subject 
drew  upon  a  core  strategy  to  handle  breakdown  situations. 

The  characterology  framework  was  applied  to  the  examination 
of  these  core  strategies.  This  lead  to  explanations  for  the 
differences  in  the  subjects*  strategies  and  the  similarity  in  each 
subject’s  strategic  approach  to  the  incidents.  Four  of  the  six 
characterologies  (identified  by  *  in  Table  1)  emerged  from  the 
data,  which  was  fortuitous.  These  are  summarised  in  Table  2. 


operational  functions.  The  cognitive  architecture  explored  in 
this  paper  focuses  on  the  strategic  behaviours  and  barriers  that 
emerge  at  breakdown  points.  In  breakdown  situations 
behaviours  emerge  which  are  intuitive  and  automatic.  When 
this  happens  a  strategy  ceases  to  be  effective,  however, 
individuals  will  continue  with  the  strategy  despite  signs  it  is 
no  longer  working.  The  underlying  theory  is  that  by  bringing 
this  subconscious  behaviour  to  the  forefront  of  conscious 
awareness,  we  are  able  to  interrupt  the  automatic  behavioural 
process.  This  leads  to  more  appropriate  and  effective 
strategies  in  breakdown  situations. 


These  findings  lead  us  to  conclude  that  the  identification  of 
dominant  character  orientation  provides  us  with  a  basis  for 
developing  hypotheses  about  the  subconscious  strategic 
behaviours  that  will  emerge  in  certain  breakdown  situations. 
This  can  make  an  important  contribution  to  our  understanding 
of  the  human  requirements  for  future  cognition  adaptive 
systems. 

3.  COGNITIVE  ARCHITECTURES  FOR  ADAPTIVE 
SYSTEM  SUPPORT 

The  research  findings  have  formed  the  basis  for  considering 
cognitive  architectures  to  support  pilots’  strategic  behaviours. 
It  is  our  theory  that  there  is  potential  for  cognition  adaptive 
systems  to  provide  pilot  support  through  structuring 
information,  providing  control  feedback  and  by  handling  basic 


Our  research  highlighted  the  interruption  of  two  major 
functions  in  breakdown  situations.  (Ref  2)  The  clarity  function 
is  interrupted  by  an  insight  barrier  and  the  effectiveness 
function  by  an  action  barrier.  These  two  functions  which  are 
essential  to  situational  appropriate  behaviours,  are  inhibited  by 
these  barriers.  The  cycle  for  situational  appropriate  behaviour 
begins  with  the  clarity  function.  Clarity  is  derived  from 
awareness,  attention  and  information.  When  an  insight  barrier 
emerges,  the  clarity  to  move  to  the  next  function  will  be 
absent  and  the  individual  will  continue  to  seek  clarity.  The 
effectiveness  function  is  interrupted  by  a  response  barrier. 
When  this  barrier  emerges,  the  individual  will  experience 
difficulty  in  responding  with  appropriate  and  effective  action. 
This  process  is  illustrated  in  Figure  1. 


Figure  1:  Cycle  for  Situational  Appropriate  Behaviour 
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Figure  2:  A  Cognitive  Architecture 
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3.1  A  Cognitive  Architecture 
The  interruption  of  the  two  primary  functions  led  us  to 
consider  a  cognitive  architecture  based  on  the  two  breakdown 
behaviours:  the  processing  of  information  and  taking 
appropriate  action.  The  cognitive  architecture  presented  in 
Figure  2  provides  a  framework  for  identifying  the 
interventions  and  support  required  by  individuals  who 
experience  these  breakdowns.  In  circumstances  when  multiple 
strategies  are  drawn  upon,  for  example,  a  breakdown  in  clarity 
shifts  to  a  response  breakdown,  by  tracking  the  individual’s 
strategic  process,  an  adaptive  system  could  switch  to  the 
appropriate  architecture. 

4.  A  MODEL  FOR  COGNITION  ADAPTIVE  SYSTEM 
SUPPORT  OF  STRATEGIC  BEHAVIOURS 
The  model  for  supporting  the  cognitive  architecture 
presented  in  Figure  3  is  drawn  from  the  Psycognition 
methodology.  It  is  based  on  the  process  an  expert  analyst 
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would  apply  in  working  with  character  strategy  in  a 
breakdown  situation.  This  process  was  mapped  on  to  a  pilot 
breakdown  scenario  drawn  from  the  research,  to  determine 
how  it  could  be  applied  to  pilots’  strategic  behaviours.  Parts  of 
the  process  and  certain  interventions  were  found  to  be 
potentially  applicable  to  the  fighter  pilot  domain. 

4.1  A  Scenario  -  Cognition  Adaptive  System  Support  for 
Mr.  Safety 

Inside  the  cockpit :  An  information  overload  situation  leading 
to  a  perceived  control  breakdown:  a  plethora  of  information, 
inside  and  outside  of  the  cockpit.  Mr.  Safety’s  internal 
dialogue  is  running,  “How  can  I  avoid  being  my  own  worst 
enemy?  Are  my  priorities  right?  I’m  losing  it.’’  He  is  suffering 
from  distraction,  “Why  isn’t  that  indicator  light  changing?  Is 
this  information  important,  can  I  trust  it?  Where  is  the  bad 
guy,  enemy  radar  is  squirting  something,  I  must  avoid  that 
storm!’’ 


Figure  3:  A  Model  for  Cognition  Adaptive  System  Support  of  Strategic  Behaviours 
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The  cognition  adaptive  system  ‘knows*  the  cognitive 
architecture  of  Mr.  Safety.  It  has  a  map  of  his  behavioural 
strategies  for  handling  a  breakdown  in  the  clarity  function. 

The  system  will  track  his  difficulties  in  processing  information 
and  will  attempt  to  manage  the  process  through  interventions 
to  reduce  the  insight  barrier  and  to  restore  the  clarity  function. 
The  system  aims  to  slow  things  down  for  Mr  Safety  so  that 
he  can  understand  the  meaning  of  the  information  he  is 
receiving.  It  attempts  to  break  things  into  smaller,  more 
manageable  steps.  It  will  also  tiy  to  keep  Mr.  Safety  in  contact 
with  the  situation  to  prevent  him  from  withdrawing  into 
confusion  or  over  analysis.  The  system  does  this  by  tracking 
signs  of  the  strategy  not  working,  which  will  be  reflected  in 
increased  confusion  and  fragmentation.  Examples  of  the 
system  tracking  process  and  interventions  are  illustrated  in 
Figures  4  and  5. 

5.  COGNITIVE  REQUIREMENTS  FOR  ADAPTIVE 
SUPPORT  SYSTEMS 

•  The  locus  of  control  must  be  appropriately  balanced 
between  the  pilot  and  the  cognition  adaptive  system.  In 
situations  where  the  locus  of  control  lies  with  the  system, 
the  pilot’s  internal  authority  must  remain  in  control. 


•  The  pilot  must  know  which  of  the  system’s  functions  s/he 
can  override  and  which  ones  they  cannot. 

•  The  system  must  support  and  not  inhibit  rational 
inientionality.  (“It  doesn’t  make  sense  to  me,  but  you  must 
have  a  good  reason.*’) 

•  The  system’s  interventions  and  behaviour  must  not 
contribute  to  the  information  overload  or  control  breakdown. 
This  will  require  close  monitoring  of  the  system  impact  on 
the  pilot  to  determine  when  the  interventions  are  helpful  and 
when  they  are  contributing  to  the  problem. 

•  The  relationship  between  the  cognition  adaptive  system  and 
the  pilot  needs  to  be  well  established  outside  of  the  cockpit. 

•  The  cognition  adaptive  system’s  understanding  of  the  pilot’s 
cognitive  architecture  and  behavioural  suategies  needs  to  be 
built  up  over  a  period  of  time.  Initially,  this  understanding 
would  be  developed  outside  of  the  cockpit.  However,  it  is 
essential  that  it  is  developed  further  through  adding 
behavioural  information  collected  during  each  mission. 

•  There  must  be  a  high  level  of  compatibility  between  the 
system  and  the  pilot  This  depends  on  the  system  supporting 
the  appropriate  cognitive  architecture  and  on  it’s  ability  to 
switch  to  a  more  appropriate  architecture  if  the  pilot’s 
behavioural  strategy  changes. 


Figure  5:  Mr.  Safety  Intervention  Scenario:  Control  Breakdown  &  Information  Overload 


Managing  the  Process: 

Organise  the  speed  by  slowing  down  &  focusing. 

Simple  precise  &  prioritising  interventions. 

Intervention  &  Feedback  Function 

Take  a  moment,  slow  down  &  focus. 

What  would  help  you  to  regain  control  right  now? 

It  seems  like  more  information  at  the  moment  is  not  useful. 

What  kind  of  infonnation  would  be  useful  to  you  right  now?* 
You  have  all  the  infonnation  you  need  for  the  moment. 

Let’s  stop  &  consider  what  the  priorities  are. 

You’re  doing  fine,  slow  down  and  focus. 

This  is  what  is  important. 

loots:  naming  the  system 

Focus:  present  &  needs 

Contact:  maintain  contact 

Prioritise:  important  information 

loots:  naming  the  system 

Prioritise:  priorities 

loots:  system  naming 

Support:  choice 

Handing  over:  Support  Function 

I'm  keeping  an  eye  on  this  so  you  don't  have  to. 

Pilot  Validation  &  Acceptance  Criteria 

Let  me  handle  the  distractions  so  that  you  can  focus. 

I’ll  keep  my  eye  on  the  red  light  so  you  don’t  have  to. 

I’ll  tell  you  when  the  fuel  level  changes. 

I’ll  remind  you  when  to  turn  the  radio  back  on. 

I’ll  take  control,  you  check  the  plan. 

Will  I  be  more  or  less  in  control? 

Do  I  need  to  or  can  I  trust  the  system? 

Can  I  trust  the  system? 

Can  I  trust  the  system? 

Can  I  tmst  the  system? 

Offers  three/four  categories  of  information:  e.g. 

Pilot  pushes  button  to  select  one:  ground  control,  weather,  electronic  warfare,  status  of  the  cockpit. 
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6.  DESIGN  IMPLICATIONS 

An  important  implication  of  the  cognition  adaptive  systeni 
scenario  is  the  uncertainty  around  the  behaviours  that  could 
result  from  the  system  interventions.  Although  the  system 
intention  is  to  evoke  appropriate  behaviours,  instead  it  could 
lead  to  an  increase  in  inappropriate  behaviours. 

In  the  scenario  described  above,  the  pilot’s  behaviour  could  be 
accentuated  instead  of  diminished.  For  example,  his  reactions 
could  lead  to  increases  in  delay,  disorganised  action, 
fragmentation,  confusion  and  impulsive  action. 
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System  interventions  could  also  inadvertently  trigger  core 
belief  behaviours.  For  example,  resistance,  power,  control, 
safety  and  trust.  These  implications  need  to  be  carefully 
researched. 


The  interpersonal  dynamics  between  the  system  and  the  pilot 
will  determine  how  effectively  a  system  can  provide  cockpit 
support  and  guidance.  Therefore  the  dynamics  that  could 
develop  from  different  system  interventions  need  to  be 
carefully  researched. 
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1.  SUMMARY 

Despite  substantial  technological  success,  “associate”  systems 
continue  to  suffer  from  a  fundamental,  sociological  prob¬ 
lem — namely,  that  human  operators  of  advanced  automation 
are  used  to  being  in  charge.  We  have  recently  begun  adapting 
techniques  from  our  associate  research  to  the  construction  of 
“tasking”  interfaces  which  enable  the  kind  of  interaction  op¬ 
erators  are  used  to  having  with  intelligent,  informed  subordi¬ 
nates.  Instead  of  being  autonomous,  or  even  truly  mixed- 
initiative,  “tasked”  systems  are  always  subordinate — ^but  they 
know  enough  about  the  tasks  in  the  domain  that  instructing 
them  is  vastly  easier  than  instructing  traditional  automation 
systems.  We  use  a  shared  task  model  to  give  advanced  auto¬ 
mation  systems  (e.g.,  Unmanned  Air  Vehicles — ^UAVs)  the 
same  task  and  goal  understanding  that  the  human  has.  When 
combined  with  a  planner,  the  resulting  system  permits  ‘task¬ 
ing’  at  all  the  various  levels  an  intelligent  subordinate  should 
be  able  to  accept:  exhaustively  specified  plans  to  be  per¬ 
formed  exactly,  partial  plans  which  leave  the  planner  free  to 
create  any  full  plan  which  includes  those  pieces,  constraints 
which  require  the  planner  to  stay  away  from  certain  methods 
or  resources,  or  high-level  goals  for  the  automation  to  achieve 
however  it  thinks  best.  The  net  result  is  a  human-machine 
system  which  is  almost  as  capable  as  an  associate,  and  which 
reduces  human  workload  almost  as  much,  yet  which  leaves 
the  human  more  in  control  than  an  associate  does. 


2.  INTRODUCTION— THE  PROBLEM 

In  the  approximately  12  years  since  the  first  associate  systems 
were  conceived  and  development  work  was  begun,  workers 
in  the  field  of  human-electronic  crewmember  systems  have 
achieved  an  impressive  array  of  technologicsJ  successes. 
Numerous  working  prototypes  have  been  demonstrated  and 
are  beginning  to  find  their  way  into  common  use.  The  field 
has  expanded  far  beyond  its  initial  beginning  with  associates 
for  fighter  pilots,  to  encompass  commercial  aviation,  auto¬ 
mobile  driving,  oil  refinery  control  room  applications,  etc. 
The  U.S.  Army  is  on  the  verge  of  flight  testing  its  Rotorcraft 
Pilot’s  Associate  and  even  Microsoft’s  latest  Office”^^  prod¬ 
ucts  are  shipping  with  a  task-based,  intent-tracked  help  sys¬ 
tem. 

In  spite  of  these  technological  achievements,  experience  has 
consistently  shown  that  the  concept  of  a  human-electronic 
crewmember,  as  it  has  been  implemented  to  date,  suffers  from 
a  basic  sociological  problem.  Namely,  pilots  (and  other  hu¬ 
man  operators  of  complex  systems)  want  to  remain  in  charge 
of  the  equipment  they  use.  In  developing  the  Rotorcraft  Pi¬ 
lot’s  Associate  Cockpit  Information  Manager  [1],  we  inter¬ 
viewed  a  variety  of  rotorcraft  pilots  and  RPA  designers  to 
develop  a  consensus  list  of  prioritized  goals  for  a  “good” 
cockpit  configuration  manager.  Two  of  the  top  3  items  on  the 


list  were  “Pilot  remains  in  charge  of  task  allocation”  and 
“Pilot  remains  in  charge  of  information  presented.” 

By  definition  [2],  and  for  good  reasons,  an  associate  system 
shares  responsibility,  authority  and  autonomy  over  many 
cockpit  behaviors  with  the  human  operator(s).  This  is  espe¬ 
cially  true  of  the  traditional  associate  behaviors  of  informa¬ 
tion  management  and  adaptive  automation.  It  is  important  to 
remember,  however,  that  the  motivation  for  creating  associate 
systems  which  had  the  capability  to  share  these  tasks  with  the 
operator  has  always  been  to  reduce  operator  workload  ,  and 
information  overload  [3].  While  operators  wish  to  remain  in 
charge,  and  it  is  desirable  that  they  do  so,  the  simple  fact  is 
that  in  today’s  complex  systems,  operators  cannot  be  fully  in 
charge  of  ^1  system  operations — certainly  not  in  the  same 
way  they  have  been  in  earlier  cockpits  and  workstations. 

Conceptually,  the  problem  can  be  presented  as  in  Figure  1. 
This  figure  shows  the  relationship  between  the  adaptiveness 
of  a  human-machine  system  as  a  function  of  the  workload  or 
unpredictability  it  causes  for  the  human  operator. 


Figure  1 .  Conceptual  view  of  the  relationship  between  sys¬ 
tem  adaptiveness,  human  workload  and  unpredictability  to 
the  human  operator. 

The  implication  of  this  view  is  that  for  any  increase  in  adap¬ 
tiveness  (that  is,  the  ability  of  the  human-machine  system  to 
perform  in  an  appropriate,  context-dependent  manner  in  dif¬ 
ferent  situations)  there  must  be  an  accompanying  increase  in 
either  human  workload  (the  amount  of  physical,  attentional  or 
cognitive  “energy”  the  human  must  exert  to  use  the  system) 
or  in  unpredictability  for  the  human  operator  (inability  of  the 
human  to  know  what  the  automation  will  do  at  any  given 
time).  Since  adaptiveness  is  generally  the  goal  of  added 
complexity  (though  systems  can  be  complex  without  achiev¬ 
ing  this  goal),  this  is  equivalent  to  saying  that  any  increase  in 
human-machine  system  complexity  must  affect  the  human 
operator  in  two  ways — either  (1)  the  added  complexity  must 
be  fully  controlled  by  the  human,  resulting  in  increases  in 
human  workload,  or  (2)  the  added  complexity  must  be  man¬ 
aged  by  automation,  resulting  in  increases  in  unpredictability 
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to  the  human.  These  alternatives  represent  endpoints  on  a 
spectrum;  many  intermediate  points  are  possible. 

Associate  systems  have  chosen  to  address  the  problems  in¬ 
curred  by  increased  adaptiveness  by  adding  an  extremely 
sophisticated  suite  of  cockpit  automation.  Thus,  they  are  near 
the  “automation”  end  of  the  spectrum  of  solutions.  When 
successful,  they  do  obtain  the  benefit  of  greatly  enhanced 
system  adaptiveness  with  little  or  no  increase  in  human 
workload.  But  they  continue  to  encounter  the  socio- 
technological  hurdle  of  diminished  control  and  increased 
unpredictability  for  the  human  operator. 

In  the  remainder  of  this  paper,  we  present  initial  work  on  a 
solution  which  allows  human  operators  to  interact  with  ad¬ 
vanced  automation  at  a  variety  of  levels.  While  this  does  not 
eliminate  the  dilemma  presented  in  Figure  1,  it  mitigates  it  by 
allowing  operators  to  flexibly  choose  various  points  on  the 
spectrum  for  interaction  with  their  automation.  We  call  hu¬ 
man-machine  systems  of  this  sort  “tasking”  interfaces,  be¬ 
cause  they  allow  posing  a  task  to  automation  at  all  the 
different  levels  one  might  ‘task’  an  intelligent,  knowledgeable 
subordinate.  The  notion  of  tasking  gives  the  operator  the 
same  type  of  control  s/he  currently  has  over  tasks  delegated  to 
subordinates,  thus  it  provides  him  or  her  with  a  greater  degree 
of  control  than  a  full  mixed-initiative  associate  system  does, 
while  allowing  low  workload  interactions  in  situations  where 
they  are  desirable. 


3.  PROPOSED  SOLUTION—  TASKING 
INTERFACES 

Figure  2  presents  our  general  architecture  for  tasking  inter¬ 
faces. 


Tasking 


Playbook 

Instructions 

Mission 
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Event 
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Algorithms 

Provably  correct  plans 
Provably  safe  reactions 


System^ontrol 


Figure  2.  General  Architecture  for  Tasking  Interfaces. 

The  primary  components  are  a  Graphical  User  Interface 
(GUI)  and  a  Mission  Analysis  component  which  are  based  on 
and  conununicate  with  each  other  and  with  the  human  op¬ 
erator  via  a  Shared  Task  Model.  The  human  operator  com¬ 
municates  tasking  instructions  in  the  form  of  desired  goals, 
tasks,  partial  plans  or  constraints  in  accordance  with  the  task 
structures  defined  in  the  shared  task  model.  These  are,  in 
fact,  the  methods  used  to  communicate  commander’s  intent  in 
current  training  approaches  for  U.S.  battalion  level  com¬ 
manders  [4].  The  Mission  Analysis  component  is  a  projective 
planning  system  which  is  capable  of  understanding  these 
instmctions  and  (a)  evaluating  them  for  feasibility  and  b) 
expanding  them  (where  necessary)  to  produce  fully  executa¬ 
ble  plans.  Once  an  acceptable  plan  is  created,  it  is  passed  to 
an  Event  Handling  component  which  is  itself  a  reactive  plan¬ 
ning  system  capable  of  making  moment  by  moment  adjust¬ 
ments  to  the  plan  to  enable  execution.  The  event  handling 
component  then  passes  these  instructions  to  control  algo¬ 
rithms  which  actually  effect  behaviors  in  the  controlled  sys¬ 


tem  automation.  Since  the  Playbook  GUI,  Mission  Analysis 
component  and  the  Shared  Task  Model  are  components 
unique  to  the  construction  of  a  tasking  interface,  as  well  as 
the  focus  of  our  current  research,  they  will  be  described  in 
more  detail  in  separate  subsections  below. 


3.1  Shared  Task  Model 

The  enabling  technology  for  a  tasking  interface  is  the  ability 
to  code,  track  and  dynamically  modify  the  plans,  goals,  tasks 
and  objectives  which  one  or  more  humans  may  have  in  oper¬ 
ating  their  automation  and  equipment.  By  explicitly  repre¬ 
senting  such  a  “task  model”  in  a  format  that  is  both  familiar 
and  recognizable  by  a  human  operator  and  which  is  interpret¬ 
able  by  a  knowledge-based  planning  system,  we  gain  a  level 
of  coordination  between  human  and  system  beyond  what  was 
previously  possible.  Work  on  the  U.S.  Air  Force’s  Pilot’s 
Associate  pioneered  such  task  models  in  two  different  for¬ 
mats:  a  plan-goal  modeling  technique  [5]  and  a  Task  Network 
representation  based,  ultimately,  on  PERT  chart  notations 
used  in  business  planning  [6]. 


These  modeling  techniques  share  several  critical  attributes,  as 
illustrated  in  Figure  3.  First,  they  are  organized  via  hierarchi¬ 
cal  decomposition — meaning  that  beneath  each  task  or  goal 
the  next  lower  layer  represents  alternate  methods  of  achieving 
that  goal,  down  to  some  bottom  layer  of  executable,  primitive 
actions.  Models  have  both  breadth  and  depth.  Their  breadth 
is  the  range  of  task  domain  which  they  cover.  A  task  model 
for  interactions  with  a  weapon  system  is  “narrower”  than  a 
model  which  covers  all  flight  operations.  Their  depth  is  a 
measure  of  the  degree  of  decomposition  or  “granularity”  on 
the  lowest  level  of  actions  represented — ^thus  a  model  of 
flight  operations  which  only  represents  major  flight  phases 
(e.g-,  take  off,  ingress,  attack,  etc.)  is  “coarser”  than  one 
wlhch  decomposes  these  actions  still  further. 


Task  Model 

Represents  all  tasks  to  be  performed  with  system 
-  variable  granularity,  hierarchical  decomposition 
May  represent  corresponding  goal  hierarchy 
Partially  ordered,  sequential  dependencies  represented 
Start,  stop,  shed,  deadline,  instantiatable  parameters,  etc.  represented 
Special  purpose  information  linked  to  tasks: 

~  information  requirements,  authorized  actors,  resource  needs,  etc. 


"‘Mission''  Model 

Represents  a  planned,  instantiated  path  through  Task  Model 


Model  Tracker 

Dynamically  tracks  (and  updates)  Mission  Model  for  current  tasks 
and  future  plans 

May  infer  tasks  from  world  states  (Le.,  sensor  inputs)  ot  rely  on  user 
inputs  (or  any  mix)  .  - 


'  Task  Sensitive  Applications 


Operator 

Information 

Management 


Resource 

Automation 

Workload 

Management 

Managemertt 

Management 

Management 

Figure  3.  The  various  uses  and  instantiations  of  task  models 
in  associate  systems. 

Task  models  have  both  static  and  dynamic  variations.  In  their 
static  variation,  they  list  all  possible  tasks  that  can  be  per¬ 
formed  in  this  human-machine  system  (to  some  level  of 
depth).  From  among  this  static  set  of  possible  tasks,  certain 
tasks  can  be  flagged  and  combined  a  priori  as  the  expected 
states  for  some  portion  of  the  future  (e.g.,  the  mission  plan). 
Static  mission  models  may  then  be  “tracked”  or  updated  dy- 
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namically  during  operation  to  represent  the  current,  unfolding 
set  of  tasks  the  human-machine  system  is  actually  engaged 
in — ^both  indicating  when  various  tasks  in  the  mission  plan 
become  active  and/or  when  the  human  operator  has  chosen  to 
do  perform  some  task  which  was  not  explicitly  planned  for 
this  mission  (e.g.,  react  to  threats). 

The  tasks  represented  in  the  model  are  (or  should  be)  drawn 
from  the  way  operators  think  about  tasks,  goals,  etc.  in  their 
domain.  They  may  be  drawn  from  training  materials  or  op¬ 
erator  interviews.  In  any  event,  the  resulting  model  should 
be  easily  readable  by  a  trained  operator  in  the  domain.  But 
tasks  in  the  model  must  be  interpretable  by  all  automation 
systems  which  will  use  them  (something  like  a  football  team's 
playbook — each  player  knows  what  he  is  supposed  to  do 
when  a  given  play  is  activated  and  coordination  is  an  emer¬ 
gent  function  stemming  from  their  all  sharing  the  same  play- 
book).  Thus,  by  ‘calling  a  play’  (that  is,  declaring  that  a 
certain  task  is  to  be  accomplished),  the  operator  can  task 
automation  subsystems  to  behave  appropriately  for  the  per¬ 
formance  of  that  task. 

In  the  creation  of  our  tasking  interface,  we  have  extended  the 
operator’s  ability  to  ‘call  plays’  by  providing  them  the  ability 
to  interact  directly  with  the  task  model,  activating  tasks  at 
various  levels  of  decomposition.  This  capability  is  provided 
via  the  Playbook  GUI  described  in  the  next  section  below. 
We  have  also  provided  a  planning  system  which  is  capable  of 
understanding  the  operator’s  commands  and  either  evaluating 
them  for  performability  or,  when  they  are  provided  at  a  level 
higher  than  is  executable  by  automation,  of  developing  an 
executable  plan  which  obeys,  yet  fleshes  out,  the  operator’s 
instructions.  This  Mission  Analysis  Component  is  described 
in  the  following  subsection. 


of  interfaces.  The  specific  attributes  of  the  GUI  will  be 
driven  by  the  uses  to  which  it  will  be  put.  As  a  simple  exam¬ 
ple,  a  pre-mission  planning  tool  will  require  much  more 
flexibility  and  precision  in  visualizing  and  interacting  with 
the  emerging  plan,  while  an  in-flight  tasking  tool  will  be  con¬ 
strained  to  use  something  as  simplified  as  a  highly-attenuated 
menu  of  only  those  tasking  options  which  are  relevant  at  the 
current  moment.  Another  factor  which  is  becoming  obvious 
in  our  initial  development  activities  is  that,  while  a  direct 
presentation  of  the  task  model  (as  is  illustrated  in  Figure  4) 
generally  provides  the  most  detailed  and  precise  interaction 
with  the  emerging  plan,  it  is  neither  the  most  familiar  nor  the 
most  efficient  presentation  for  many  pilot  planning  tasks. 
Interaction  with  the  underlying  model  via  a  map-based  pres¬ 
entation  and  a  simple  timeline  view  are  both  being  considered 
to  address  this  problem. 


Figure  4.  One  possible  instantiation  of  a  playbook  GUI. 


3.2  Playbook  Graphical  User  Interface 

The  human  operator  must  have  a  method  of  inspecting  and 
interacting  with  the  task  model,  both  to  understand  the  possi¬ 
ble  actions  which  could  be  taken  to  achieve  known  goals  and, 
more  importantly,  to  declare  those  tasks,  goals,  partial  plans 
and  constraints  s/he  wishes  the  system  to  pursue.  This  inter¬ 
face  must  provide  the  operator  with  a  method  of  “calling 
plays”  as  described  above.  Through  this  interface  the  opera¬ 
tor  (e.g.,  pilot)  will  graphically  construct  a  full  or  partial  plan 
for  the  mission  specifying  the  tasks  (or  “plays”)  to  be  per¬ 
formed  and  goals  to  be  accomplished.  Some  requirements  for 
the  Playbook  GUI  we  are  constructing  include:  (1)  the  set  of 
“plays”  (e.g.,  maneuvers,  procedures,  etc.)  represented  will  be 
those  that  any  well-trained  operator  should  know  (thus  mak¬ 
ing  it  intuitive  and  easy  to  learn  and  use),  (2)  the  general  play 
‘templates’  in  the  playbook  can  be  composed  and  instantiated 
to  create  any  specific  mission  plan,  and  (3)  The  operator  may 
select  plays  to  be  used  at  various  levels  in  the  hierarchical 
decomposition  of  the  mission  plan,  leaving  the  remainder  to 
be  selected  and  composed  by  the  Mission  Analysis  Compo¬ 
nent  (see  below),  either  requiring  or  prohibiting  the  use  of 
specific  plays.  This  makes  true  mixed  initiative  planning  a 
possibility  and  will  make  the  tasking  of  a  UAV  much  like 
providing  instructions  to  a  human  wingman. 

A  hypothetical  example  of  a  Playbook  GUI  is  presented  in 
Figure  4.  While  we  are  only  beginning  the  development  of 
this  tool,  some  factors  are  already  clear.  First,  the  underlying 
links  to  a  task  model  both  permit  and  require  a  wide  variety 


3.3  Mission  Analysis  Component 

The  Mission  Analysis  Component  (MAC)  integrates  projec¬ 
tive  planning  technology  with  a  human  tasking  mechanism. 
MAC  operates  over  full  or  partial  mission  plans  provided  via 
the  playbook  GUI,  performing  two  sub-functions:  (1)  ana¬ 
lyzing  die  operator’s  plan  for  feasibility  and  goal  achievement 
by  means  of  verifying  constraints  and  (2)  automatically  com¬ 
pleting  partial  mission  plans  (or  suggesting  candidate  com¬ 
pletions)  in  keeping  with  the  requirements  and  prohibitions 
imposed  by  the  pilot.  The  hierarchical,  typed  representation 
of  plays  in  the  playbook  simplifies  choices  for  plan  comple¬ 
tion  by  limiting  the  types  of  plays  which  are  useful  at  each 
level.  Plays  contain  information  about  their  expected  and 
desired  effects.  This  information  is  used  by  projection  algo¬ 
rithms  in  the  MAC  to  analyze  the  plan  for  correctness  (i.e., 
will  it  achieve  its  stated  goals  if  executed  successfully?). 
Constraint  propagation  techniques  are  used  to  coordinate  the 
individual  plays  chosen  by  the  pilot  or  the  system  into  coher¬ 
ent  mission  plans  (e.g.,  once  a  target  is  stipulated  for  one 
phase  of  the  plan,  it  is  propagated  to  the  other  phases  auto¬ 
matically).  The  MAC  module  we  are  developing  draws  on 
work  in  AI  planning,  but  must  be  part  of  an  approach  bridg¬ 
ing  the  traditional  gap  between  analysis  and  execution. 

Analytic  capabilities  developed  in  previous  AI  planners  have 
used  simpler,  more  abstract  task  representations  and  have 
made  substantial  assumptions  about  correct  execution.  Reac¬ 
tive  planning  and  execution  systems  (such  as  those  used  in 
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robots)  are  essentially  high  level  programming  languages 
used  to  coordinate  the  behaviors  and  states  of  sensors,  effec¬ 
tors  and  to  handle  contingencies — ^but  without  support  for 
analysis  of  correctness.  Honeywell  is  concurrently  at  work 
on  an  integrative  architecture  for  projective  planning,  reactive 
planning,  and  control  actuation,  called  CIRCA  [7]  which 
bridges  this  gap  by  synthesizing  provably  correct  reaction 
programs  in  accordance  with  known  goals.  To  date,  however, 
little  thought  has  been  given  to  how  an  operator  will  provide 
those  high-level  goals.  Our  work  fills  that  gap. 

3.4  Integration  and  Operation 

At  the  bottom  layer  of  our  architecture,  advanced  control 
algorithms  provide  for  actual  moment-to-moment  vehicle 
control.  The  control  algorithms  provide  several  levels  of 
functionality.  The  highest  level  is  the  ability  to  fly  specific 
flight  segments  (e.g.  close  or  loose  formation,  "pop-up"  for 
weapon  release,  rendezvous).  At  a  lower  level  are  guidance 
functions  like  waypoint  steering  and  terrain  following.  Fi¬ 
nally  at  the  lowest  level  are  the  attitude  and  rate  stabilization 
control  functions.  The  resulting  “plan”  created  jointly  by  the 
human  operator  using  the  Playbook  GUI  and  by  the  MAC’S 
reviewing  or  fleshing  out  the  human’s  instructions,  along 
with  reactive  adaptations  provided  by  the  event  handling 
component,  gives  these  control  functions  the  instmctions  they 
need  to  know  which  behaviors  to  execute.  The  result  is  a 
seamless  ability  for  the  operator  to  task  and  control  the  auto¬ 
mation  functions  in  a  wide  variety  of  ways  depending  on  the 
human’s  available  time  and  degree  of  trust. 


4.  A  TASKING  INTERFACE  FOR  UAVs 
We  have  recently  begun  work  on  a  proof-of-concept  demon¬ 
stration  of  this  tasking  interface  architecture.  We  have  cho¬ 
sen  to  work  in  the  domain  of  Unmanned  Air  Vehicles 
(UAVs)  both  because  this  domain  is  of  interest  to  Honeywell 
and  its  customers  and  because  the  tasking  of  UAVs  by  human 
operators  already  engaged  in  highly  demanding  activities 
(e.g.,  aircraft  or  helicopter  pilots)  is  a  good  example  of  the 
situation  depicted  in  Figure  1  above.  Current  and  emerging 
UAV  interfaces  either  require  operators  to  remotely  control 
the  aircraft  via  a  dedicated  cockpit  mockup,  or  they  rely  on 
very  high  level  behaviors  (e.g.,  Close  Air  Patrol  circuits  or 
waypoint-designated  routes)  which  can  be  commanded  but 
not  modified.  The  first  approach  provides  a  high  degree  of 
predictability  and  adaptiveness,  but  only  at  the  cost  of  very 
high  operator  workload;  the  second  approach  minimizes  op¬ 
erator  workload  and  provides  highly  predictable  behavior,  but 
only  at  the  cost  of  adaptiveness.  Neither  of  these  approaches 
is  sufficient  for  placing  one  or  more  UAVs  at  the  disposal  of 
an  operator  who  is  concurrently  engaged  in  the  piloting  of  his 
own  aircraft.  To  make  coordinated  operations  between  a 
human  pilot  and  UAV(s)  feasible  demands  increases  in  adap¬ 
tiveness  without  substantial  increases  in  either  unpredictabil¬ 
ity  or  pilot  workload. 

Building  on  prior  control  algorithms  and  simulation  work 
supporting  a  scenario  of  one  piloted  F-16  with  two  unmanned 
F-16  “wingmen”,  we  have  begun  developing  a  tasking  inter¬ 
face  to  enable  a  human  pilot  to  lay  out  a  mission  plan  for  the 
UAV’s.  This  interface  will  support  the  stipulation  of  goals, 
partial  plans,  fiill  plans  and  constraints  for  each  of  the  UAVs 
either  separately  or  in  conjunction.  We  have  begun  our  work 


by  concentrating  on  a  ground-based  tasking  interface  due  to 
its  lighter  demands  on  pilot,  simulation  and  interface  design. 
However,  we  believe  that,  with  suitable  modifications  to  the 
GUI,  this  approach  will  be  suited  to  in-flight  tasking  as  well. 

To  date,  we  have  used  a  variation  on  the  PERT  chart-based 
task  network  representation  as  pioneered  by  McDonnell- 
Douglas.  A  portion  of  this  representation  for  the  high-level 
task  of  “ground  attack”  is  presented  in  Figure  4.  While  defi¬ 
nition  of  the  Playbook  GUI  is  proceeding  at  this  time,  we  will 
step  through  a  representative  example  of  interaction  between 
the  human  operator,  the  Playbook  GUI  and  the  Mission 
Analysis  component  below. 

The  human  “leader”  of  the  pilot  -i-  UAVs  team  (presumably 
the  pilot  himself)  would  interact  with  the  tasking  interface  to, 
first,  declare  that  the  “mission  task”  for  the  day  was  “Ground 
Attack.”  Having  declared  only  that  much  to  a  team  of  trained 
human  pilots,  the  team  would  have  a  very  good  general  pic¬ 
ture  of  the  mission  they  would  be  flying;  similarly,  our  inter¬ 
face  (via  the  MAC)  knows  what  a  typical  ground  attack  plan 
consists  of.  But  just  as  a  human  leader  instructing  a  flight 
team  could  not  leave  the  instructions  at  that,  so  the  human 
“tasker”  is  required  to  provide  a  bit  more  information  to  in¬ 
stantiate  and  bind  the  high  level  task.  In  this  case,  the  tasker 
must  provide  at  least  the  specific  target  of  the  ground  attack. 
S/he  could  provide  substantially  more  detail  (such  as  take  off 
time,  route,  munitions,  roles  for  the  wingman,  etc.)  but  s/he 
could  also  hand  the  task  off  to  intelligent  team  members  at 
this  point  and  let  them  work  out  the  best  plan  they  can  come 
up  with.  The  tasker  can  do  this  as  well,  using  our  interface  to 
hand  the  task  to  the  MAC  with  only  this  information.  For  our 
example,  we  will  assume  that  the  tasker  wishes  to  provide 
more  plan  specifications. 

Both  the  tasker  and  the  interface  know  that  any  Ground  At¬ 
tack  plan  must  consist  of  Ingress,  Target  Attack  and  Egress 
subtasks,  in  that  order.  It  may  also  include  a  Defense  Sup¬ 
pression  task  which  would  run  in  parallel  with  Target  Attack. 
These  relationships  are  depicted  in  the  task  network  in  Figure 
4:  conditional  branches  are  indicated  by  diamonds,  sequential 
relationships  are  depicted  left  to  right  and  parallel  tasks  are 
depicted  next  to  each  other  vertically  with  a  round  “join” 
node  showing  that  both  parallel  tasks  must  be  completed 
before  the  next  task  can  begin.  Figure  4  also  shows  the  tasker 
selecting  among  alternative  methods  of  performing  the  “De¬ 
fense  Suppression”  task. 


Figure  5.  Hypothetical  route  plan. 


Assume  that  the  tasker 
wishes  to  provide  de¬ 
tailed  instructions  about 
how  the  Ingress  task  is 
to  be  performed.  His 
instructions  are  pre¬ 
sented  graphically  in 
Figure  5.  The  tasker 
wishes  the  whole  team 
to  fly  Nap  of  the  Earth 
(NOE)  from  waypoint  1 
to  2  and  then  to  split, 
with  the  two  UAVs 
flying  a  Low  Observ¬ 


able  (LO)  route  to  waypoint  3a  and  on  to  4  (perhaps  serving 
as  a  decoy),  while  the  pilot  will  fly  NOE  to  waypoints  3b  and 
on  to  rejoin  at  4. 
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In  order  to  task  the  UAVs  to  perform  in  this  fashion,  the 
leader  must  begin  by  expanding  the  Ingress  task  in  the  plan 
developed  in  Figure  4  above.  Upon  doing  so,  he  would  re¬ 
ceive  the  “generic”  Ingress  task  representation  shown  in  Fig¬ 
ure  6. 


Figure  6.  Expansion  of  Ingress  Task. 


It  is  important  to  note  that  this  is  not  a  default  method  of  do¬ 
ing  “Ingress”  so  much  as  it  is  a  generic,  uninstantiated 
method — corresponding  loosely  to  what  a  human  operator 
knows  about  how  Ingress  can  or  should  be  done  in  general. 
That  is,  the  trained  pilot  knows  that  in  order  to  accomplish 
Ingress,  they  will  need  to  all  Take  Off,  will  probably  need  to 
Assemble,  will  certainly  need  to  Fly  to  Objective  and  then 
Prepare  for  Split  (assuming  they  assembled  in  the  first  place). 
Fly  to 

Objective  r 


HjFly  to  Waypoint  2^ 

5all 


“I  Fly  Air  Corridor  [■ 


Fly  to  Waypoint  |  - 1 

ftUAVI  &UAV2  „  I 


Waypoint  #: _ 3n 

Waypoint  location:  F4. 
Time:  (opt)  _ 


Actor:  UAV1&UAV2 


Figure  7.  Expansion  of  Fly  to  Objective  for  second  way- 
point,  UAVs  only. 

For  our  example,  weTl  assume  that  the  leader  doesn’t  wish  to 
place  constraints  on  how  the  Take  Off  and  Assemble  tasks  are 
to  be  performed  (that  is,  he  will  leave  the  planning  of  these 
tasks  up  to  the  MAC),  but  that  he  does  wish  to  mandate  that 
the  “Fly  to  Objective”  task  be  performed  in  a  specific  way. 
We  are  using  a  dotted  line  convention  in  these  figures  to  indi¬ 
cate  no  constraints  imposed,  by  either  human  or  MAC,  on  the 
conduct  of  this  task  and  grey  lines  plus  shading  to  indicate 
that  the  pilot  has  imposed  constrains.  Tasks  are  represented 
by  boxes,  and  the  actor(s)  associated  with  the  task  are  indi¬ 
cated  in  the  lower  left  comer  of  the  box.  Figure  6  also  illus¬ 
trates  a  pop  up  window  via  which  the  tasker  can  input 
required  information  for  the  task. 

If  the  pilot  expands  the  Fly  to  Objective  task  to  begin  to  input 
his  route  constraints,  he  first  gets  a  “generic”  method  of  per¬ 
forming  the  task — as  illustrated  in  Figure  7.  This  diagram 
says  that  one  can  fly  to  an  objective  by  doing  one  or  more 
(note  the  optional  loop)  Fly  to  Waypoint  or  Fly  Air  Corridor 
tasks.  In  Figure  7,  the  pilot  has  chosen  a  Fly  to  Waypoint 
task  and  is  entering  required  information  to  indicate  that  all 
actors  will  fly  to  waypoint  2  at  location  D3. 


Since  the  MAC  realizes  that  the  performance  of  this  Fly  to 
Waypoint  task  does  not  yet  accomplish  the  parent  task’s  goal 
of  reaching  the  objective  (that  is,  of  being  at  H6),  MAC  in- 
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Figure  8.  Expansion  and  stipulation  of 
Fly  to  Objective  Task. 

structs  the  GUI  to  present  another  set  of  the  generic  options  it 
knows  are  available  to  accomplish  the  goal.  The  developing 
task  network  is  redrawn  as  in  Figure  8.  Here  the  first  “Fly  to 
Waypoint”  task  is  included  as  a  stipulated  part  of  the  plan, 
but  the  same  set  of  choices  is  presented  for  the  remaining 
flight  segments,  since  the  system  knows  that  this  is  how 
“flying  to  an  objective”  can  be  accomplished.  Again,  the 
tasker  could  choose  to  leave  the  remaining  specification  to 
the  MAC— in  which  case,  MAC  would  develop  a  plan  which 
incorporated  the  first  flight  or  report  that  this  was  impossible. 
Instead,  however,  the  tasker  goes  on  to  stipulate  how  the  next 
segment  is  to  be  flown,  indicating  that  the  two  UAVs  are  to 
fly  to  waypoint  3a  by  themselves. 
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Figure  9.  Expansion  of  Fly  to  Waypoint  to  flight  parameters. 


If  the  tasker  continues  to  stipulate  the  set  of  waypoints  indi¬ 
cated  in  the  hypothetical  route  plan  in  Figure  5  above,  the 
resulting  task  network  would  look  like  Figure  9,  with  separate 
branches  for  the  pilot  and  the  two  UAVs  for  waypoints  3  and 
4.  Figure  9  also  illustrates  the  further  expansion  of  the  Fly  to 
Waypoint  tasks  to  illustrate  how  the  tasker  can  impose  con¬ 
straints  on  low  level  details  about  how  the  flying  is  to  be 
done.  By  decomposing  the  “Fly  to  Waypoint”  task  down  still 
further,  the  tasker  can  access  attributes  of  the  flying  task; 
flight  formation,  system  configuration,  flight  method,  etc. 
Under  each  of  these,  there  is  a  series  of  “Achieve”  and  “Fly” 
tasks  which  correspond  to  various  options  about  how  flight 
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may  be  performed.  We  have  expanded  the  “Achieve  Flight 
Method”  branch  to  illustrate  the  stipulation  of  NOE  flight  for 
all  actors  for  the  first  flight  segment.  The  remaining  seg¬ 
ments  could  be  stipulated  similarly.  Note,  though,  that  the 
tasker  has  made  no  constraints  on  the  remaining  flight  pa¬ 
rameters,  leaving  MAC  free  to  plan  these  as  needed  to  ac¬ 
complish  higher  level  mission  tasks  or  obey  other  imposed 
constraints.  Note  too,  that  the  flight  “behaviors”  depicted  at 
the  lowest  level  in  Figure  9  are  now  at  the  highest  level  that 
the  control  algorithms  for  UAVs  are  currently  capable  of 
executing.  Thus,  we  have  reached  the  level  at  which  model 
development  can  reasonably  end. 

5.  DIRECTIONS  FOR  FUTURE  GROWTH 

We  are  still  in  the  design  stages  of  our  proof  of  concept  task¬ 
ing  interface,  but  we  are  already  identifying  methods  for  im¬ 
provement.  First  among  these  is  the  need  to  evaluate 
usability.  We  intend  to  allow  pilots  and  engineers  with  mili¬ 
tary  mission  planning  experience  to  review  our  prototype,  but 
a  thorough  evaluation,  ideally  in  comparison  with  traditional 
tasking  techniques,  would  be  more  appropriate. 

Although  the  focus  of  our  initial  research  has  been  the  devel¬ 
opment  of  the  task  model  infrastructure  and  methods  for  hu¬ 
man  and  computer  sharing  of  it,  experience  indicates  that  we 
need  several  interface  improvements.  First,  we  must  integrate 
the  task  model  view  (presented  above)  of  the  plan  with  alter¬ 
nate  views  including  a  map-based  tool  (to  provide  better  ori¬ 
entation  and  a  more  familiar  method  of  creating  mission 
plans),  and  a  timeline  view  to  provide  better  visualization  of 
the  temporal  layout  of  mission.  Second,  we  need  to  enable 
rapid,  simple  interaction  with  the  model  at  multiple  levels — 
taskers  will  frequently  wish  to  specify  a  general  plan  with  one 
very  detailed  constraint  (e.g.,  ‘runway  attack,  but  don’t  use 
rockeyes,’  but  with  the  current  interface  they  can  only  reach 
that  constraint  by  stepping  through  multiple  layers  of  a  de¬ 
tailed  plan.  Third,  operators  will  wish  to  indicate  that  many 
tasks  are  to  be  done  the  same  way  (e.g.,  fly  all  waypoints  after 
crossing  the  FEBA  in  NOE)  but  at  present  they  must  input 
this  constraint  separately  for  each  task.  We  are  now  explor¬ 
ing  the  use  of  an  object-hierarchy  of  tasks  as  a  means  of  ap¬ 
plying  a  constraint  to  multiple  tasks  within  a  class.  Finally, 
we  need  a  simple  method  of  dictating  negatives  to  the 
MAC — that  is,  stating  that  any  method  of  accomplishing  the 
task  is  acceptable  except  this  one. 

While  we  plan  to  address  the  above  issues,  further  necessary 
developments  are  beyond  our  current  scope:  (1)  Our  current 
task  network  representation  is  weak  in  its  coding  of  goals, 
and  we  believe  that  goals  are  a  critical  component  of  any 
tasking  interaction  [4].  Future  work  should  explore  repre¬ 
sentations  which  mix  the  sequential  strengths  of  the  task  net¬ 
work  with  the  goal-to-plan  relationships  of  a  plan-goal  graph. 
(2)  Tasking  interfaces  should  not  rely  on  a  pre-defmed  set  of 
task  models.  The  operator  should  be  able  to  create  novel 
tasks  and  to  store  components  of  models  which  are  useful . 
This  indicates  the  need  for  a  compositional  tool  and  a  library 
of  stored  models.  (3)  We  believe  that  a  “run-time”  (e.g.,  in 
flight)  version  of  the  tasking  interface  is  feasible  and  desir¬ 
able.  Such  an  interface  would  necessarily  support  a  more 
narrow  range  of  operator  controls  and  modifications  of  the 
model.  It  should  provide  more  rapid  access  to  a  tightly  con¬ 
strained  set  of  alternate  plans  or  modifications  of  high-level 


parameters  within  existing  plans.  To  facilitate  transfer  of 
training,  however,  it  should  maintain  at  least  some  of  the  look 
and  feel  of  the  ground-based  tool. 

6.  CONCLUSIONS 

Our  approach  to  tasking  interfaces  is  intended  to  build  a 
bridge  from  current  systems  to  true  “associates”.  While  pilots 
are  rarely  comfortable  giving  over  authority  to  automation  at 
all  times  and  situations,  they  are  willing  to  have  it  around  for 
those  times  when  it  may  be  useful.  Tasking  interfaces  are  a 
method  of  allowing  the  pilot  to  remain  fully  in  control,  yet  of 
enabling  almost  the  full  autonomy  of  an  associate  to  plan  and 
execute  a  high-level  task  whenever  the  pilot  deems  that  that 
level  of  assistance  is  appropriate.  The  net  result  is  a  human- 
machine  system  which  is  imost  as  capable  as  an  associate, 
and  which  reduces  human  workload  almost  as  much,  yet 
which  leaves  the  human  more  in  control  than  an  associate 
does.  Perhaps  by  requiring  (and  enabling)  an  associate  to 
behave  more  like  an  intelligent  subordinate,  pilots  will  be 
more  tolerant  of  their  weaknesses  and  more  willing  to  let 
them  show  their  capabilities  in  tightly  controlled  settings. 
Through  use,  then,  pilots  may  become  more  familiar  with 
their  strengths  and,  ultimately,  more  willing  to  tolerate  them 
on  a  roughly  equal  footing  in  the  cockpit. 
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1  SUMMARY 

Due  to  increasing  demands  put  on  crews  of  military  aircraft, 
effective  cockpit  systems  will  be  required  in  order  to  reduce 
workload  and  to  improve  aew  performance.  This  paper 
presents  an  approach  to  crew  assistance  in  tactical  flight 
missions.  The  underlying  tasks  are  tactical  decision  making, 
low-level  fliglU  planning  and  flight  guidance.  The  Tactical 
Situation  System  as  a  knowledge-based  crew  assistant  It 
includes  the  capability  of  situation  assessment  and  planning 
as  well  as  a  crew  interface  and  a  flight  guidance  display  with 
sensor  and  s>Tithctic  vision  components.  The  Tactical 
Situation  System  offers  a  promising  solution  to  improve  the 
situational  awareness  of  the  crew.  A  software  prototype  has 
been  successfully  tested  and  evaluated  in  a  simulated 
environment  as  w^ell  as  by  flight  trials. 


2  INTRODUCTION 

Human  operators  might  be  overtaxed  due  to  tlie  various  tasks 
related  to  military  air  transport  missions  in  hostile 
environments.  Guiding  tlic  aircraft  through  adverse  weather 
conditions  in  ground  proximity  puts  further  demands  on  crew 
performance.  Most  accidents  can  be  at  least  partly  attributed 
to  human  erroneous  actions  due  to  increasing  crew  workload. 
[3]  Humaii-ccnlered  automation  [1]  oilers  a  promising 
approach  to  the  solution  of  the  obvious  problems  of  those 
design  philosopliies  in  flight  deck  automation  just  utilizing 
tlie  available  technology  regardless  of  human  needs. 

The  main  scope  of  present  programmes  on  on-board  pilot 
assistance  is  tlie  cnlianccment  of  the  crew’s  situational 
awareness.  Situational  awareness  is  guaranteed  if  the  pilot 
has  all  relevant  information  for  the  present  flight  situation  at 
his  disposal  and  is  therefore  able  to  cope  with  the  posed 
tasks.  In  order  to  improve  the  situation  assessment 
capabilities  it  must  be  ensured  tliat  the  attention  of  the 
cockpit  crew  is  continuously  guided  towards  the  objectively 
most  urgent  task  of  tlie  situation  and,  if  necessary,  the 
workload  reduced  to  a  normal  degree  which  can  be  handled 
by  the  crew.  [3] 

In  order  to  meet  the  above  design  principles  of  the  human- 
centered  approach  an  appropriate  crew  assistant  s>'stem  has  to 
incorporate  the  capabilities  of  situation  assessment  and 
planning  in  the  field  of  tactical  air-operations.  The  situation 
assessment  has  to  be  perfomed  by  the  cockpit  crew 
continuously  during  flight.  In  parallel  the  same  assessment 
should  be  done  by  the  functions  of  the  machine  part  of  the 
man-machine  system. 


Additionally,  the  visual  perception  aspects  of  human 
performance  have  to  be  considered.  Enhanced  and  s>Tithetic 
vision  systems  are  today’s  solution  in  order  to  improve  the 
crew’s  situational  awareness  in  the  context  of  low-level  flight 
guidance.  While  classical  flight  director  systems  avoid  or  at 
least  decrease  the  involvement  of  the  pilot, 
enhanced/synthetic  vision  systems  keep  the  pilot  active  in  the 
flight  guidance  loop.  Tlus  is  achieved  by  depicting 
information  suitable  for  each  human  performance  level  [4] 
instead  of  just  appealing  to  the  skill-based  level  as  required 
for  aircraft  stabilization  and  control.  Thereby,  the  principles 
of  the  cognitive  approach  to  flight  deck  automation  can  be 
met. 

This  paper  describes  the  design  and  functions  of  the  Tactical 
Situation  System  representing  a  military  operations  related 
assistant  system  extended  by  the  addition  of  an  Enhanced 
Flight  Guidance  Display  System  to  assist  the  pilot  in  manual 
visually  guided  flight  at  low  altitudes  and  in  adverse  weather 
conditions. 


3  THE  TACTICAL  SITUATION  SYSTEM 

Tlie  Tactical  Situation  System  is  a  software  protot>pe  system 
developed  utilizing  spin-otT  effects  from  the  actinties  of  the 
Crew  Assistant  Military  Aircraft  (CAM.A)  [7],  Enhanced 
Vision  System  (EVS)  [5]  and  Future  Large  Aircraft  Night  and 
Adverse  Weather  Vision  (FLA-NSWS)  [6].  Primarily  the 
military  operations  related  aspects  of  crew'  assistance  are 
covered.  The  aim  of  the  investigations  is  to  create  flexible 
sy'stcm  prototypes  for  cockpit  avionics  in  order  to  elaborate 
user  requirements  and  evaluate  respective  prototypes  witli 
operational  personnel  under  human-machine-interaction 
considerations.  Advanced  mission  management  technologies 
are  demonstrated  in  order  to  support  the  pre-development 
phase  of  future  air  transport/w^eapon  systems. 

3.1  Approach  to  tactical  flight  crew  assistance 
While  performing  a  tactical  low-level  flight  mission, 
deviations  from  a  preplanned  trajectory  might  be  induced  by 
the  crew  while  reacting  to  a  suddenly  changing  mission 
scenario.  Under  adverse  weather  conditions  this  creates  a 
high  crew  workload  and  a  loss  of  situational  aw'areness 
concerning  the  aircraft’s  position  and  attitude  relative  to  the 
terrain  and  the  desired  flight  trajectory.  The  suggested  crew^ 
assistant  system  is  the  Tactical  Situation  System  [5].  It  yields 
the  capability  of  taking  workload  off  the  crew  by  giving 
decision  aids  while  keeping  up  the  crew’s  situational 
a\\'areness.  This  is  achieved  by  the  integration  of  the 
following  functional  capabilities: 
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•  Situation  interpretation  and  assessment  through  terrain 
and  threat  analysis, 

•  Planning  through  on-board  mission  management  and 
optimal  trajectory  generation, 

•  Situation  visualization  through  enhanced  and  synthetic 
vision. 

The  following  section  contains  a  closer  view  to  the 
architecture  and  functions  of  the  Tactical  Situation  System. 

3.2  Functions  of  the  Tactical  Situation  System 

The  Tactical  Siuation  System  is  designed  as  an  on-board 
cockpit  system  with  interfaces  to  the  crew  via  cockpit 
displays  and  controls,  to  the  aircraft  systems,  and  to  ground 
stations  via  data  link.  Based  on  the  current  tactical  scenario 
the  Tactical  Siuation  System  performs  a  situation  assessment 
resulting  in  a  danger  and  threat  analysis.  The  latter  provides 
the  input  for  the  flight  planning,  by  computing  an  optimal 
trajectory  in  terms  of  survival  probability  according  to  the 
crew-given  mission  constraints.  The  trajectory  is  visualized 
on  various  cockpit  displays  such  as  a  sjnthetic  vision  primary 
flight  display  and  an  advanced  tactical  map  navigation 
display.  Additionally  the  planned  flight  path  can  be  issued  to 
an  aircraft-hosted  flight  guidance  system  in  order  to  generate 
flight  director  commands  or  perform  automatic  flight  control. 
Respective  flight  guidance  symbology  is  fused  with  a  sensor 
image  and  the  latter  superimposed  by  the  an  enhanced  vision 
overlay  for  display  on  a  cockpit  head-up/level  equipment 


Figure  1:  Modules  and  integration  of  Tactical  Situation 
System 

Figure  1  showes  the  modules  of  the  Tactical  Situation  System 
and  a  suggested  integration  into  existing  aircraft  avionics  as 
used  for  the  simulator  trials  described  in  section  4.1.  The 
following  subsections  provide  a  closer  insight  into  the 
functional  details  of  the  Tactical  Situation  System’s  core 
modules. 

3. 2, 1  The  interacti ve  tactical  situation  map  display 

The  Tactical  Situation  Display  provides  the  primary  crew 
interface  to  the  Tactical  Situation  System.  Basically,  it  is  an 
interactive  electronic  moving  map  display  for  navigational 
and  operational  purposes. 

The  aim  of  the  interface  is  the  improvement  of  the  pilot’s 
situational  awareness  by  depicting  situation  relevant  data. 
The  actual  information  needed  for  task  performance  is  highly 
influenced  by  the  task  itself  Obviously,  the  map  information 
required  for  EFR  flight  is  completely  different  to  the 
information  in  the  context  of  tactical  navigation.  Typically, 
the  differences  are  more  subtle.  The  Tactical  Situation 
Display  is  primarily  designed  in  order  to  cope  with  the 
advancing  Imowledge  on  human  information  processing. 
Therefore,  it  allows  the  composition  of  the  display  contents 
from  a  vector  oriented  database.  The  di^Iay  utilizes  digital 
terrain  elevation  data  (DTED)  and  digital  feature  analysis 
data  (DFAD)  [10]  in  order  to  create  a  topographical  map  in 
any  required  scale  and  orientation.  The  various  feature 
classes  can  be  displayed  selectively  providing  a  very  efficient 
decluttering  of  the  screen  contents.  Radio  navigation 
symbology  can  be  added  by  visualizing  Jeppesen  Navigation 
Data.  Military  operations  related  aspects  are  covered  by  the 
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incorporation  of  tactical  symbols.  Furthermore,  the 
interpreted  tactical  situation  is  dynamically  depicted  utilizing 
a  three-dimensional  threat  coverage  diagram.  The  threat  map 
display  can  be  activated  for  the  different  above  ground  levels 
or  be  slaved  to  the  aircraft’s  present  altitude.  The  mission 
plan  and  an  optimized  ground  track  of  the  low-altitude  flight 
plan  is  indicated  as  in  Figure  4. 


Figure  2:  Tactical  Situation  Display 


Figure  2  tries  to  give  a  general  impression  of  the  appearance 
of*  the  Tactical '^Situation  Display.  It  shows  the  display 
depicting  the  feature  data  and  the  ground  elevation  map  in  a 
1:250,000  north-up  representation.  The  system  is  designed  as 
a  head-down  display. 

The  second  main  aspect  of  the  Tactical  Situation  Display  is 
on  the  side  of  control.  It  provides  an  interface  to  the 

•  Low-altitude  flight  planner.  It  allows  the  pilot  to 
enter/edit  waypoints  interactively  by  marking  them  on  the 
map; 

•  Tactical  situation  assessment  by  supporting  the 
manipulation  of  tactical  elements. 

The  control  is  done  by  use  of  a  cockpit  trackball  for  the 
analogue  inputs  and  pushbuttons/touchscreen  for  the  discrete 
inputs. 

This  concept  fuses  the  functionalities  of  the  navigation 
display  with  interactive  elements  derived  from  a  flight 
management  system  control  and  display  unit  by  creating  a 
bidirectional  interface.  The  inte^ated  function  is  supposed  to 
be  situated  on  a  primary  flight  display.  Thereby,  the 
unnatural  separation  of  flight  plan  manipulation  and 
depiction  in  current  cockpits  can  be  resolved. 

The  following  two  subsections  describe  the  background 
modules  of  the  Tactical  Situation  System  which  are 
controlled  by  the  Tactical  Situation  Display  and  provide  the 
actual  assistance  functions  in  terms  of  situation  assessment 
and  planning. 

J,  2. 2  The  tactical  situation  interpretation 
The  Tactical  Situation  Interpreter  is  a  knowledge-based 
module  in  the  context  of  the  situation  assessment.  Its  main 
contribution  is  the  computation  of  a  threat  mpp  which  gives  a 
penalty  value  distribution  over  the  considered  operation  area. 

Hostile  or  threatening  tactical  elements  are  passed  to  the 
Tactical  Situation  System  via  a  data  link  connection  or  can  be 
entered  into  the  system  through  the  Tactical  Situadon 
Display.  The  tacdcal  elements  themselves  provide  little 
informadon  to  the  crew  in  terms  of  decision  aids.  The  tactical 
situation  interpretadon  now  assesses  the  tacdcal  elements  in 


the  context  of  knowledge  about  the  surrounding  world,  the 
threat’s  characteristics  and  the  ownship’s  capabilities  and 
situation. 

The  calculations  are  based  upon  digital  terrain  elevation  data 
[10]  and  the  threat’s  models.  Threats  such  as  surface-to-air 
missiles  (SAM)  or  radar  sites  are  described  by  a  set  of  t>pical 
parameters  such  as  maximum  range,  operationability, 
efficiency  along  range  and  respective  models  for  threat  area 
overlapping.  Aeronautical  constraints,  restrictions,  and 
tactical  considerations  can  be  incorporated  as  well. 

Due  to  the  characteristics  of  the  threat’s  radar  systems  and 
respective  radar  shadows  resulting  from  the  terrain  structure, 
the  altitude  above  ground  up  to  which  an  aircraft  is  not 
detectable  by  the  hostile  radar  beams  can  be  derived  from  the 
digital  terrain  elevation  database  (DTED).  [8] 


Figure  3:  Threat  map  with  SAMs*  range  circles 


Figure  3  depicts  a  typical  result  of  a  threat  map  calculation. 
The  result  of  the  tactical  situation  assessment  provides  a 
powerful  decision  aid  and  a  strong  enhancement  of  situational 
awareness.  The  crew  is  enabled  to  identify  the  threat-free 
corridor  in  middle  of  the  selected  scene. 

3.2.3  The  Low-altitude  Flight  Planner 
The  Tactical  Situation  System  allows  the  planning  of  a  low- 
level  flight  mission  interactively  by  use  of  the  Tactical 
Situation  Display  (as  shown  in  Figure  4).  Relevant  mission 
plan  information  such  as  heading,  distance,  timing  are 
updated  and  displayed  on-line.  The  waypoint  planning  is 
done  with  respect  to  the  given  mission  constraints. 


Figure  4:  Mission  planning  with  the  tactical  map 


The  aim  of  the  Low-altitude  Flight  Planner  is  the  calculation 
of  a  three-dimensional  route  between  the  mission-given 


waypoints  with  a  maximum  probability  of  survival  in  a 
hostile  environment  This  is  achieved  by  avoiding  threatened 
areas  if  possible,  minimizing  the  exposure  to  unknown 
threats  and  keeping  clear  of  the  terraiit  Therefore,  the 
mission  constraints,  the  tactical  elements  and  the  resulting 
threat  map,  the  terrain  elevation  data  and  the  aircraft 
performance  data  are  taken  into  consideration.  The  output  of 
the  planner  is  a  detailed  trajectory  and  a  waypoint/flightleg- 
oriented  representation  Figure  5  shows  the  architecture 
concept  of  the  planner. 


Figure  5:  Low-altitude  Flight  Planner  architecture 


The  sy’stem  consists  of  three  main  functional  submodules: 

1.  The  danger  analysis  incorporates  the  threat  map 
calculation  as  described  in  subsection  3.2.2.  Additionally, 
the  visibility  at  each  point  is  calculated  without  assuming 
any  particular  threats.  The  algorithm  issues  lower  danger 
values  on  the  side  of  valleys  than  in  the  center.  This 
behariour  reflects  pilot’s  low-level  flying  preferences. 
Finally,  the  danger  analysis  utilizes  the  calculation  of  a 
ground  collision  probability,  which  is  particularly  high  in 
rough  terrain.  This  feature  leads  to  generally  higher  flight 
altitudes  in  the  absence  of  threats.  An  overall  penalty 
value  is  calculated  for  each  terrain  grid  point  and  stored 
as  a  danger  model  in  an  array. 

2.  The  moding/control  checks  the  flight  status  and 
assembles  the  target  point  and  the  planning  area  for  the 
optimization  according  to  the  mission  constraints.  Tlie 
numerical  optimization  is  based  on  dynamic  programming 
[2].  The  optimization  provides  an  array  of  optimal 
directions  to  the  target  point.  As  long  as  a  replanning 
does  not  imply  a  new  target  point  another  optimization 
run  is  not  required.  This  means  that  the  algorithm  offers 
an  optimal  path  from  each  point  in  the  planning  area  to 
the  desired  target  point.  This  peculiarity  of  the  algorithm 
provides  a  most  powerful  assistant  function  of  the  low- 
altitude  plaiuier:  the  rapid  in-flight  replanning  capability. 
This  function  allows  the  generation  of  a  new  optimal 
trajectory  starting  at  the  aircraft’s  present  position  within 
a  single  second.  In  the  case  of  intended  or  unvoluntary 
deviation  from  the  preplaimed  track,  a  recovery  trajectory 
can  be  issued  taking  the  terrain,  tactical  situation  and 
mission  goals  into  consideration.  The  function  can  be 
activated  intentionally  by  the  crew,  by  a  ground  proximity 
system  alert  or  any  other  appropriate  crew  assistance 
function. 

3.  The  path  selection  depends  on  the  current  planning  mode 
(initial  planning  or  replanning).  It  constructs  a  terrain 
grid  based  flight  path  from  a  given  start  point  or  the 
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present  aircraft  position  to  the  target  point.  The  output 
assembly  functions  trajectory  synthesis  and  plan  analysis 
form  the  Low-altitude  Flight  Planner  output. 

•  In  order  to  be  monitored  by  a  pilot  model  based 
assistant  system,  such  as  CANIA  [7IP]» 
representation  of  the  detailed  trajectory  has  to  be 
reduced  to  a  waypoint  based  low  altitude  flight  plan, 
which  represents  the  general  considerations  to  be 
followed  in  the  human  planning  of  low  level  missions 
i.e.  threat  avoidance,  terrain  masking,  timing  etc.  The 
reduction  is  done  through  a  low  pass  filtering 
operation  of  the  optimal  trajectory. 

•  Additionally,  the  low-level  flight  plan  is  given  by  a 
detailed  trajectory  representation.  Thereby,  an 
assisting  fimction  on  the  skill-based  human 
performance  level  can  be  provided.  During  normal 
operation  the  pilot  selects  the  trajectory  to  fly  by  the 
consideration  of  relevant  influences.  The  execution 
can  be  monitored  by  the  crew  assistant.  In  situations 
of  increased  workload  it  is  possible  that  the  pilot  is  no 
longer  able  to  select  a  safe  and  efficient  flight  path.  In 
this  case  the  display  of  the  automatically  generated 
trajectory  is  a  helpful  tool. 


Figure  6  shows  a  typical  planning  result  between  two 
waypoints  considering  only  the  terrain  elevation.  The  result  is 
an  optimal  trajectory  minimizing  the  cost  function  of 
weighted  terrain  elevation  data  and  local  threat  values 
integrated  over  the  complete  fligiit  path. 

Obviously  there  has  to  be  some  function  which  provides  crew 
assistance  on  following  the  optimal  trajectory  in  terms  of 
primary  flight  guidance.  The  following  subsection  gives  an 
overview  of  the  flight  guidance  incorporated  in  the  Tactical 
Situation  System. 

3. 2. 4  The  Enhanced  Flight  Guidance  Display 
Several  scientific  research  studies  [9]  start  from  the 
assumption  that  the  pilot’s  information  gathering  from  the 
out-the-window  view  is  critical  for  flight  guidance  in  low- 
level  flight  and  landing  tasks.  Flying  under  restricted  visual 
conditions  requires  additional  techmeal  means  to  assist  the 
pilot  in  performing  the  task.  Classical  flight  guidance  systems 
for  instrument  flight,  such  as  flight  director  display  or  ILS 
indicator,  give  only  very  reduced  information  gathered  by 
simple  sensors  from  the  real-world  situation.  Therefore,  the 
pilot’s  situational  awareness  might  be  fairly  low  and  the  pilot 
still  has  to  learn  adapted  flying  skills  instead  of  just  utilizing 
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his  natural  and  extremely  powerful  skills  of  flying  in  a  three- 
dimensional  visual  world- 

The  Enhanced  Flight  Guidance  Display  is  a  proir^g 
approach  to  the  solution  of  the  problems  of  poor  visioility 
low-level  flight.  It  comprises  an  imaging  sensor  (e.g.  FLIR, 
mmWR,  LL-TV)  and  the  superimposition  of  the  sensor  image 
with  a  computer-generated  three-dimensional  cockpit  view. 
The  incorporation  of  sensory  data  is  essential  for  the  benefit 
of  the  sy'stem.  Due  to  incomplete  or  incorrect  databases,  the 
pilot  cannot  only  rely  on  the  synthetic  image  components. 
Inaccuracies  of  the  navigation  system  yield  another  basic 
problem  for  just  synthetic  vision  systems. 


Figure  7:  Synthetic  symbology  of  Enhanced  Flight 
Guidance  Display 


Thus,  the  Enhanced  Flight  Guidance  Display  is  not  a  visual 
simulation.  It  does  not  try  to  produce  a  most  realistic 
representation  of  the  out-the-window  view  as  used  in  visual 
systems  of  training  flight  simulators  but  is  instead  a  display 
carrying  situation  and  task  relevant  information. 

Therefore,  the  Enhanced  Flight  Guidance  Display  consists 
out  of  the  following  visual  components: 

•  The  non-confomial  flight  guidance  overlay  is  a  standard 
head-up  display  symbology  with  speed,  altitude  (MSL, 
AGL),  heading,  vertical  velocity  readouts  and  a  bank  and 
sideslip  indicator. 

•  The  conformal  flight  guidance  symbology  incorporates  an 
attitude  display  and  artificial  horizon.  The  flight  guidance 
tunnel  visualizes  the  three-dimensional  flight  trajectory. 
A  velocity  vector  and  flight  path  predictor  depict  dynamic 
aircraft  movement  information.  [9]  The  predictor  symbol 
(not  used  in  flight  trials)  consists  of  three  U-shaped 
brackets  (for  the  I,  2  and  3  second  prediction)  in  order  to 
fit  into  the  flight  guidance  tunnel.  The  symbology  is 
denoted  as  tunnel  dock, 

•  The  conformal  terrain  contour  symbology  is  a 
perspective  depiction  of  the  digital  terrain  elevation 
database.  Air  traffic  obstacles  exceeding  a  minimum 
height  taken  from  the  digital  feature  analysis  database  are 
shown  as  well  as  airfields. 

•  The  sensor  image  is  added  to  the  synthetic  parts  of  the 
display  in  order  to  cope  with  incomplete  databases. 

Figure  7  tries  to  depict  the  synthetic  parts  of  the  display.  The 
format  shown  is  typical  for  a  head-down  application,  because 
of  the  coloured  terrain  surface.  The  colouring  is  switchable 
between: 

•  an  absolute  elevation  coding  colour  key  as  used  in  the 
map  display  and 


•  a  collision  warning  colour  code,  wiiich  assigns  red  colour 
to  the  terrain  higher  than  the  ownship  altitude. 

The  same  colour  codes  can  be  independently  assigned  to  a 
perpendicular  east/north-fixed  terrain  grid.  Using  only  the 
terrain  grid  ^^ithout  the  surface  colouring  produces  a  more 
head-up  display  type  symbology  which  is  applicable  in 
combination  with  the  sensor  image  (see  Figure  8). 

»  ‘  '  i  - 


Figure  8:  Flight  guidance  symbology  for  sensor  image 
combination 


The  software  prototype  of  the  Enhanced  Flight  Guidance 
Display  is  designed  in  order  to  allow  rapid  changes  of 
formats  and  functions  to  easily  meet  user  and  design 
requirements. 


4  EVALUATION  CONCEPTS 

In  order  to  evaluate  the  approach  to  crew  assistance, 
particularly  the  Tactical  Situation  System,  software 
prototypes  are  implemented  and  integrated  in  appropriate  test 
environments.  Tests  havebeen  carried  out  with  respect  to 
technical  feasibility,  human-machine-interaction 
considerations,  and  real-world  conditions.  Therefore,  the 
Tactical  Situation  System  and  various  subsystems  have 
continuously  undergone  critical  evaluation  procedures  which 
are  described  in  the  following  sections. 

4.1  Simulator  trials 

The  Tactical  Situation  System  was  designed  and  prototyped 
in  a  generic  cockpit  simulator  [5]  at  ESG  and  then  integrated 
and  evaluated  in  the  Daimler-Benz  Aerospace  DASA  Airbus 
development  flight  simulator  for  Future  Large  Aircraft  at 
Hamburg.  The  main  objective  of  the  experiment  was  to 
investigate  whether  the  crew  performing  a  tactical  flight 
mission  under  adverse  weather  conditions  could  be 
effectively  assisted  by  the  prototype  system  in  terms  of 
situational  awareness  improvement. 

4.1.1  Apparatus 

The  experimental  system  was  a  full  scale  three-seat  fixed- 
base  flight  simulator  equiped  with  a  collimated  wide  FOV 
visual  simulation  system.  The  cockpit  hosted  two  10  inch 
high  resolution  CRT  displays  for  each  crew  member  and  a 
collimated  head-up  display  for  the  pilot  flying.  Flight  control 
was  provided  by  Airbus  cockpit  controls  including  sidestick 
control.  The  crew’s  control  actions  were  passed  to  an  Airbus 
flight  control  system.  Flight  director  signals  were  provided  by 
a  low-level  flight  guidance  system. 


4. 1. 2  Subjects  and  scenario 

The  subjects  were  seven  German  Air  Force  pilots:  four  of 
them  tactical  transport  instructor  pilots,  two  test  pilots  with 
fighter  experience,  and  one  civil  transport  pilot  Each  of  them 
had  to  perform  a  tactical  low-level  transport  mission  of  about 
45  minutes.  The  mission  contained  portions  of  transit  and 
tactical  flight.  Low-level  flight  was  prformed  at  250  ft  AGL 
utilizing  terrain  masking  in  mountain  valleys.  Additional 
features  of  the  mission  were  a  tactical  drop  procedure,  an 
intended  deviation  from  the  preplanned  track  with  an 
imguided  recovery,  and  the  performance  of  an  unguided  go- 
around  pattern  at  the  destination  airfield.  The  whole  mission 
had  to  be  performed  under  night  and  low  visibility  (400 
meters)  visual  conditions. 

4. 1. 3  Evaluation  results 

One  of  the  main  objectives  of  the  described  experiment  was 
the  knowledge  elicitation  for  future  developments  in  the  field 
of  cockpit  systems  desigiL  Therefore,  a  continuous 
assessment  of  the  prototypes  was  performed  during  the 
simulated  flight  by  applying  the  method  of  observation  during 
task  performance.  Additionally,  debriefings  with 
questioimaires  were  conducted.  The  observations  showed  that 
the  preferred  configuration  of  the  Tactical  Display  was  a 
height-colour-coded  terrain  relief  with  a  collision-warning 
overlay.  This  supported  terrain  aviodance  and  the  low-level 
flight  guidance  task  extremely  well.  Crew  coordination 
aspects  were  promoted  significantly  by  the  integrated  display 
concept.  Concerning  the  head-up  display,  a  strong 
enhancement  could  be  achieved  by  adding  a  (simulated)  FLIR 
image.  The  ability  of  ego-motion  estimation  in  the  S>nlhetic 
Vision  alone  was  regarded  as  insuflicient.  Overall,  a  total  of 
seven  one-hour  low-level  flight  missions  were  successfully 
conducted  under  visual  conditions  prohibiting  unaided  terrain 
masking.  An  unguided  go-around  manoeuvre  was  performed 
successfully  three  times.  Two  touch-and-go  procedures  were 
conducted  utilizing  the  Synthetic  Vision  navigation 
capabilities. 

4.2  Flight  trials 

In  order  to  evaluate  the  Tactical  Situation  System  under  real 
world  conditions  flight  trials  were  conducted  recently. 

4. 2 A  Apparatus 

The  experimental  plattform  was  a  two  turbo-propeller  engine 
Domier  128  aircraft  provided  by  the  Technical  University  of 
Braunschweig.  It  was  equipped  with  a  hybrid  higli  precision 
differential  GPS  /  laser  INS  navigation  system. 


Figure  9:  Experimental  cockpit  display 
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The  experimental  cockpit  was  equipped  with  a  head-up 
mounted  13  inch  high  resolution  LCD  flat  panel  display. 
Figure  9  shows  the  experimental  cockpit  display  obscuring 
the  test  pilot’s  out-the-window  view.  A  Silicon  Graphics 
Indigo2  High  Impact  graphics  workstation  hosting  the 
experimental  softv\’are  was  mounted  in  a  shock-absorbing 
rack. 

4.2. 2  Subjects  and  scenario 

The  subjects  were  two  scientific  test  pilots,  one  working  as 
safety  pilot  and  the  other  as  experimental  pilot.  A  total  of  six 
low-level  flights  were  conducted  at  altitudes  of  approximately 
500  ft  AGL.  The  main  task  was  to  navigate  in  terrain 
proximity  and  perform  terrain  masking  in  mountain  valleys. 
Point  to  point  navigation  utilizing  the  visual  identification  of 
waypoints  was  to  be  conducted.  On-board  planning  and  in¬ 
flict  replanning  tasks  were  added.  The  actual  mission 
contents  were  designed  with  respect  to  familiarization  and 
training  considerations  and  became  increasingly  complicated. 

For  safety  reasons  the  flights  were  performed  under  VMC.  As 
they  were  meant  to  be  under  adverse  weather  conditions,  the 
outside  view  of  the  experimental  pilot  was  obscured. 

4.2.2  Evaluation  results 

Firstly,  the  flight  trials  were  evaluated  under  the  aspects  of 
technical  operability  of  the  assistance  functions  under  real- 
world  conditions.  It  showed  that  an  on-board  full  mission 
preparation  and  planning  could  be  conducted  including 
autonomous  trajectory  generation.  High  precision  visual 
navigation  tasks  with  waypoint  identification  were 
successfully  performed.  The  crew  was  enabled  to  cope  with 
an  in-flight  change  of  the  mission  order  by  conducting  an 
automatic  full  replanning.  Experimentally  induced  trajectory 
deviations  could  be  recovered  by  the  crew  by  use  ofthe  rapid 
in-flight  replaiming  capability  of  the  Tactical  Situation 
System.  Situational  Awareness  was  guaranteed  by  the  terrain 
visualization.  Generally  an  excellent  overall  pilot  acceptance 
could  be  gained  for  the  functions  and  formats. 

A  second  major  concern  of  the  evaluation  was  the  assessment 
of  the  pilot’s  situational  awareness  with  respect  to  certain 
situational  elements.  To  achieve  this,  questionnaires  were 
completed  by  the  pilot  after  the  flight  missions.  Generally,  it 
can  be  stated  that  the  situational  awareness  for  aircraft 
attitude,  altitude  and  speed  was  regarded  as  good. 
Concerning  tlic  ownship  attitude  or  proximity  witli  respect  to 
obstacles  and  terrain,  llic  synthetic  vision  format  could  be 
improved.  The  depiction  of  the  tasks  and  mission  progress 
was  regarded  as  ver\'  helpful. 

In  future  additional  support  of  the  navigation  system’s 
altitude  charmel  by  the  radar  altimeter  and  the  terrain 
elevation  model  would  be  advisory  for  further  experimental 
activities. 


5  CONCLUSIONS 

The  aim  of  the  presented  research  and  development  activities 
is  to  provide  advanced  cockpit  avionics  systems  improving 
the  crew’s  situational  awareness  with  respect  to  a  safe  and 
successful  mission  completion.  This  is  done  by  use  of 
technical  means  such  as  knowledge-based  on-board  systems 
in  the  context  of  human-centered  cockpit  automation.  The 
Tactical  Situation  System  yields  a  promising  approach  to 
assistance  in  tactical  flight  missions  containing  planning  and 
decision-making  tasks  as  well  as  low-level  fli^t  guidance. 
The  military  operations  related  subsystems  are  the  Tactical 
Situation  Interpreter,  the  Low-altitude  Flight  Planner,  and  the 
Tactical  Situation  Display.  The  system  is  enlarged  by  the 
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Enhanced  Flight  Guidance  Display  which  offers  visual  flight 
guidance  information  to  the  crew  for  poor  visibility  low-level 
flight  operations.  Thereby,  crew  assistance  can  be  provided 
on  each  human  performance  level  including  highly  cognitive 
planning  tasks  as  well  as  sensomotoric  flight  control  tasks.  A 
major  aspect  of  the  protot>pe  development  is  the  integration 
of  the  different  modules  in  order  to  achieve  an  efficient 
assistant  system. 

In  order  to  evaluate  the  system  under  techmeal  and  human- 
machine-interaction  aspects,  softw-are  protot>pes  are  being 
developed  and  integrated  in  a  flight  simulation  environment. 
A  particularly  successful  flight  trial  for  the  integrated 
assistant  system  has  been  completed  recently.  The  resulte 
show  that  this  approach  to  visual  fli^t  guidance  assistance  is 
extremely  powerfiil  and  yields  a  high  potential  for  further 
developments  in  the  field  of  human-centered  cockpit  design. 
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SUMMARY 

The  JOUST  facility  at  DERA  Farnborough  is  a  work-station 
based  man-in-the-loop  simulator,  capable  of  allowing  many 
pilots  to  engage  in  simulated  air  combat.  It  was  created  in 
order  to  perform  high  quality  operational  analysis  of  air 
combat  issues  using  human  pilots.  The  requirement  was  partly 
due  to  the  fact  that  existing  computer  only  models  were 
insufficiently  capable  of  representing  all  the  tactics  and  pilot 
abilities  used  in  a  modem  air  battle. 

In  the  past  six  years  more  than  fifty  trials  have  been 
conducted  using  front-line  aircrew,  tactics  experts  and  DERA 
scientists.  As  a  result  the  JOUST  team  have  considerable 
experience  in  modem  air  combat  tactics  and  software 
modelling.  The  JOUST  software  consists  of  a  number  of  high 
fidelity  models  representing  systems  such  as  the  aircraft, 
missiles,  radar  and  electronic  warfare,  along  with  software  for 
monitoring  and  controlling  the  simulation. 

Three  years  ago  work  started  on  the  development  of  an 
automated  pilot  for  the  JOUST  system  known  as  ’Joe’.  Joe's 
tactics  are  written  entirely  in  FORTRAN  77.  This  may  be 
considered  an  unusual  language  to  use  by  modem  software 
engineers  and  artificial  intelligent  experts.  However,  the 
authors  strongly  believe  that  the  major  problems  lie  with 
quantifying  tactics  into  hard  unambiguous  mles  and  that  the 
implementation  of  these  mles  in  software  is  a  relatively  trivial 
problem  in  comparison.  The  Joe  system  is  an  excellent 
environment  for  the  development  and  testing  of  tactics 
software  as  all  the  cockpit  displays  are  animated  in  exactly  the 
same  way  as  if  a  human  pilot  was  flying.  A  pilot  is  able  to 
shadow  Joe’s  decisions  throughout  a  combat  and  decide  if  he 
would  make  the  same  decisions  based  on  the  information 
presented.  At  his  present  state  of  development  Joe  is  as  good 
as  the  average  pilot  that  flies  on  JOUST,  with  his  strength 
being  the  accurate  flying  of  his  aircraft. 

A  natural  development  of  this  work  was  to  develop  Joe  into  a 
tactical  adviser  for  a  human  pilot.  A  prototype  system  has 
been  produced  and  utilises  standard  JOUST  simulator  station 
and  software,  Joe's  tactics,  and  direct  voice  input  and  output 
so  that  the  pilot  can  interact  with  the  software.  The  major 
problems  have  been  found  to  be  in  the  quality  of  Joe's  tactics 
and  this  obviously  hinders  the  type  of  advice  that  can  be  given 
to  the  pilot.  If  in  the  fullness  of  time  Joe  becomes  very  adept 
at  making  tactical  decisions,  then  we  are  faced  with  the 
philosophical  question;  do  we  actually  need  a  pilot  in  the 
cockpit?  Work  is  still  continuing  on  improving  this  system, 
along  with  looking  at  other  aspects  of  tactical  decision  aids 


such  as  the  use  of  fast  fly-out  missile  models  for  predicting 
threat  levels. 

1  INTRODUCTION 

The  current  aerial  battlefield  is  a  complex  environment 
imposing  a  high  workload  on  the  pilot.  Advances  in  sensors, 
communications  and  weapons  systems  will  all  confer  a 
greater  capability  to  future  fighter  aircraft,  but  unless  properly 
managed  will  add  to  the  pilot's  workload.  An  additional 
consideration  is  the  current  trend  in  fighter  aircraft  design  to 
move  away  from  dual  crew  aircraft  (e.g.  F3  Tornado,  F-4 
Phantom)  to  single  seat  aircraft  (e.g.  EF  2000,  F-22,  SU-35). 
To  provide  some  balance,  there  is  a  need  to  incorporate  a  high 
degree  of  computer  automation  in  any  future  cockpit. 

The  aim  of  the  work  carried  out  by  System  Integration 
Department  was  to  investigate  the  problems  associated  with  a 
computerised  tactical  advisory  system  that  could  be  used  to 
assist  pilots  engaged  in  air  combat.  The  ultimate  goal  was  to 
develop  a  prototype  tactical  adviser  (RITA  -  Real  time 
Intelligent  Tactical  Adviser)  for  use  within  the  JOUST 
simulation  system  and  demonstrate  that  it  can  increase  the 
combat  effectiveness  of  a  RAF  fighter  pilot  of  average  ability. 

The  design  of  RITA  drew  on  two  existing  areas  of  work.  The 
first  comprised  of  a  number  of  functions  built  into  the  JOUST 
weapon  system  that  were  designed  to  reduce  pilot  workload, 
and  the  second  was  a  prototype  computer-controlled  opponent 
for  the  JOUST  system  known  as  'Joe'. 

The  primary  technical  issue  was  to  decide  exactly  which 
aspects  of  an  on-going  combat  a  tactical  adviser  could  be 
most  useful.  This  was  achieved  partly  by  using  the  experience 
gained  from  the  large  number  of  air  combat  studies  which  had 
already  been  carried  out  using  the  JOUST  simulation  facility 
and  also  by  conducting  discussions  and  interviews  with  front¬ 
line  aircrew.  It  was  expected  that  the  adviser  would  have  to  be 
tailored  more  to  the  needs  of  a  relatively  inexperienced  pilot, 
rather  than  a  seasoned  fighter  pilot.  The  aspirations  people 
had  for  the  tactical  adviser  also  had  to  be  tempered  by  what 
was  practically  possible  within  the  timescales  and  current 
computational  capacity.  It  was  recognised  from  the  outset 
that,  for  the  foreseeable  future,  no  computer  is  likely  to  have 
the  tactical  intuition  of  a  well-trained  fighter  pilot. 

2  JOUST  SIMULATOR  OVERVIEW 

The  JOUST  system  is  a  workstation  based  multi-player  air 
combat  simulator.  It  was  developed  by  DERA  to  conduct 
operational  analysis  and  has  been  in  operation  since  October 
1991.  The  systems  that  are  simulated  have  been  validated 
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against  existing  reference  models  and  a  wealth  of  knowledge 
has  been  built  up  within  the  research  team.  JOUST  has  proven 
to  be  extremely  flexible  and  reliable  in  its  six  years  of 
operation.  During  this  period  many  simulated  air  combat  trials 
have  been  undertaken,  for  various  MoD,  industry  and 
overseas  sponsors.  The  research  team  is  made  up  of  ten 
scientists  and  engineers  and  one  RAF  aircrew  officer.  In 
addition  to  support  from  other  experts  in  DERA  the  facility 
has  a  close  relationship  with  various  parts  of  the  RAF,  RN 
and  UK  MoD,  including  Operational  Research  branches,  the 
Air  Warfare  Centre  the  Tornado  F3  Operational  Evaluation 
Unit  and  front  line  aircrew.  The  role  of  the  JOUST  facility  is 
to  provide  a  diverse  simulation  environment  capable  of 
supporting  aeronautical  system  trade-off  studies,  tactics 
development,  human  factors  studies  and  training  technology 
requirements  studies,  in  order  to  improve  the  combat 
effectiveness  of  fixed  wind  fighter  aircraft. 

The  architecture  of  the  JOUST  system  allows  for  two 
opposing  teams  of  multiple  manned  simulated  aircraft,  with 
the  facility  to  introduce  computer  generated  air  and  ground 
threats  (see  Figure  1).  The  cockpit  stations  can  be  of  a 
medium  fidelity  desktop  arrangement  (see  Figure  2)  or  high 
fidelity  cockpit  mock-up.  Each  cockpit  station  comprises  of  at 
least  two  Silicon  Graphics  computer  workstations.  One 
generates  the  out  of  cockpit  display  which  includes  a  basic 
representation  of  ground,  sky  and  weather,  generic  fast  jet 
format  head-up-display  and  all  the  aircraft,  vehicles,  missiles 
and  countermeasures  that  would  be  visible  to  the  pilot.  The 
other  runs  the  models  which  comprise  the  simulated  aircraft, 
the  interface  to  throttle,  stick  and  switches  and  also  generates 
the  head  down  instrumentation,  including  a  combined  radar 
and  IRST  B  scope,  a  tactical  display,  a  RHWR  display  and  a 
weapon  status  panel  (see  Figure  3). 

The  majority  of  the  JOUST  system  is  written  in  FORTRAN 
and  the  entire  system  consists  of  approximately  275,0(K)  lines 
of  code. 

3  THE  DEVELOPMENT  OF  JOE 

As  previously  mentioned  the  JOUST  facility  was  created  to 
carry  out  operational  analysis  using  men-in-the-loop. 
Historically  this  work  had  been  done  within  the  UK  using 
purely  digital  models,  that  'flew'  the  aircraft  in  the  combat. 
There  were  two  major  problems  with  this  approach: 

a.  For  a  new  aircraft  concept  it  was  hard  to  determine  the 
best  tactics  to  use,  especially  if  the  system  under 
consideration  had  both  good  and  bad  points,  for  example 
an  aircraft  with  superb  energy  agility  but  with  a  relatively 
small  fuel  load. 

b.  Having  determined  the  ideal  tactics,  it  is  veiy  hard  to  get  a 
computer  to  perform  these  tactics  in  a  sensible  and  robust 
manner. 

There  is  however  still  a  need  for  digital  simulations  as  man- 
in-the-loop  simulations  are  generally  more  expensive  and 
constrained  by  time  in  what  they  can  achieve.  Hence  a  few 
years  ago  the  development  of  a  new  digital  air  combat  model 
was  started  using  a  relatively  novel  approach. 


Existing  digital  models  such  as  ATEM  in  the  UK  and 
BRAWLER  in  the  US  are  very  large  programs  that  run  on  a 
single  computer  in  non-real  time  (either  faster  or  slower), 
simulating  all  the  players  in  a  combat.  The  approach  adopted 
here  was  to  develop  an  automated  pilot  that  could  fly  the 
existing  JOUST  simulator  system  of  networked  pilot  stations 
in  Beyond  Visual  Range  (BVR)  air  combat.  It  was  recognised 
from  the  outset  that  this  was  a  very  difficult  task  and  it  was 
very  imlikely  that  the  automated  pilot  would  ever  present  a 
serious  challenge  to  a  good  fighter  pilot.  Hence  rather  than 
choose  a  pretentious  name  for  the  automated  pilot,  he  was 
simply  given  the  name  Joe,  short  for  Joe  Numbnuts. 

The  software  architecture  of  a  Joe  station  is  almost  identical 
to  that  of  a  manned  station,  with  the  only  exception  being  that 
rather  than  call  the  routines  that  read  the  hardware,  Joe's 
tactics  logic  is  called  instead.  In  order  to  fly  his  aircraft  Joe 
has  to  manipulate  the  control  stick  in  an  identical  manner  to  a 
human  pilot,  for  instance  banking  and  then  pulling  back  on 
the  stick  in  order  to  turn. 

There  are  a  number  of  advantages  to  this  approach,  the 
greatest  being  that  it  is  an  excellent  environment  to  develop 
and  judge  tactics.  In  a  conventional  digital  model  the 
automated  pilots  behaviour  is  generally  only  judged  by 
observation  using  some  form  of  God's  eye  display.  In  the  Joe 
approach  all  the  pilot  displays  (radar,  RHWR,  tactical,  HUD 
etc)  animate  in  exactly  the  same  way  as  if  a  pilot  was  flying. 
When  implementing  tactics  the  developer  can  'shadow'  fly  the 
JOUST  station,  looking  for  all  the  cues  that  would  lead  him  to 
make  a  tactical  decision  if  he  was  actually  flying  the  station. 
To  judge  Joe's  behaviour  the  developer  is  able  to  see,  via  the 
head  down  displays,  all  the  information  that  Joe  is  basing  his 
decisions  on.  Very  often  a  decision  which  looks  odd  when 
observed  from  a  God's  eye  display,  is  perfectly  reasonable 
when  you  see  Joe's  sensor  displays. 

Another  major  advantage  of  this  approach  is  that  it  uses 
existing,  well  tested  software  models.  This  is  obviously  an 
advantage  for  financial  reasons,  but  it  is  also  a  major 
advantage  for  validating  the  models.  A  man-in-the-loop 
simulator  is  a  much  more  demanding  environment  when 
assessing  the  robustness  of  the  models  than  a  purely  digital 
simulation.  If  a  pilot  perceives  a  deficiency  or  bug  in  the  radar 
model  for  instance,  such  as  the  radar  failing  to  lock  when  he 
expects  it  to,  he  will  call  attention  to  the  fact.  In  a  purely 
digital  simulation  a  situation  like  this  may  easily  be 
overlooked,  as  there  is  no  pilot  there  to  complain.  As  Joe's 
software  models  are  used  in  exactly  the  same  manner  as  they 
have  been  used  in  thousands  of  hours  of  man-in-the-loop 
flight  simulation,  it  is  reasonable  to  have  confidence  in  them. 

An  advantage  for  a  facility  that  already  has  manned 
simulators  of  this  type,  is  that  the  simulation  computers  can 
be  put  to  good  use  at  times  when  they  would  normally  be 
dormant,  such  as  overnight  and  at  weekends.  Obviously  if  a 
research  group  only  wanted  to  do  digital  simulations,  this 
would  be  a  very  expensive  approach  because  of  the  large 
number  of  computers  it  requires  (at  least  one  for  each 
combatant). 

Another  disadvantage  is  that  Joe  is  constrained  to  mn  in  real 
time.  The  primary  reason  for  this  is  the  computing  power 
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required  but  also  if  the  models  were  run  faster  than  real  time  it 
might  highlight  software  bugs  that  are  not  critical  in  the 
manned  version  of  the  software  when  running  at  real  time. 

Joe's  tactics  consider  the  following  aspects  of  BVR  air 
combat: 

a.  Situational  awareness. 

b.  Sensor  management. 

c.  Energy  management. 

d.  Judgement  of  danger  level. 

e.  Co-operative  tactics. 

Joe's  tactics  are  written  entirely  in  FORTRAN  and  although 
this  may  seem  very  'quaint  and  out-dated’  by  modem  software 
engineers  and  artificial  intelligent  experts,  there  are  some  very 
good  reasons  why  FORTRAN  is  as  good  as  any  other 
approach.  Firstly  most  of  the  so  called  Intelligent  Knowledge 
Based  Systems  (IKBS)  languages  encountered  by  the  authors 
have  little  or  no  intelligence.  They  generally  break  down  into 

constructs  similar  to  FORTRAN  If.. ...then else  statements, 

although  this  is  not  always  apparent  at  first. 

Although  IKBS  languages  do  offer  advantages  such  as  mle 
tracers,  editors  etc.  the  authors  strongly  believe  that  the  major 
problems  lie  with  quantifying  tactics  into  hard  unambiguous 
mles  and  that  the  implementation  of  these  mles  in  software  is 
a  relatively  trivial  problem  in  comparison.  For  instance,  take  a 
relatively  simple  mle  such  as  'when  in  danger  mn  away'. 
Danger  has  to  be  defined  as  a  combination  of  range  to  a 
target,  do  I  think  the  target  can  see  me,  do  I  think  the  target 
wants  to  engage  me  in  a  multi-bogey  scenario,  am  I  inside  the 
targets  weapon  envelope  etc.  Run  away  has  to  be  defined  as 
what  heading  do  I  turn  onto  (for  example  a  reciprocal  heading 
or  one  towards  friendly  forces),  how  much  'g'  do  1  pull  during 
the  turn,  do  I  dive  to  gain  speed  in  the  short  term  or  do  I  climb 
to  the  tropopause  to  gain  speed  in  the  long  term.  The  mle  has 
to  be  stmetured  so  that  it  is  not  called  when  the  most 
appropriate  action  to  reduce  the  danger  is  to  try  to  shoot  down 
the  opponent.  The  mle  has  to  be  undertaken  with  a  certain 
amount  of  decisiveness,  but  not  too  much  otherwise  it  will 
inhibit  a  more  appropriate  mle  from  being  carried  out  when 
the  situation  changes  significantly.  Sorting  out  all  of  the 
above  problems  can  often  take  80-90%  of  the  time,  with  the 
coding  and  testing  taking  a  mere  10-20%. 

Other  good  practical  reasons  for  using  FORTRAN  within  the 
JOUST  facility  are : 

a.  The  staff  are  all  current  in  FORTRAN 

b.  The  tactics  software  interface  easily  with  the  models,  that 
are  themselves  written  in  FORTRAN 

c.  There  is  good  software  support  and  the  compilers  are 
relatively  bug  free. 

Joe's  tactics  currently  consist  of  approximately  4,000  lines  of 
FORTRAN  code,  not  including  comments  and  blanklines. 


only  one  way  to  judge  the  approach  taken  and  that  is  how 
good  is  the  artificial  pilot  in  combat? 

To  judge  Joe’s  combat  performance  the  JOUST  ranking 
system  has  to  be  first  introduced  (see  Figure  4). 

A  typical  JOUST  trials  pilot  will  be  of  Squadron  Leader  rank 
and  come  from  an  air  defence  background.  During  a  week  of 
training  he  will  progress  to  the  'Dangerous'  category.  After 
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about  two  or  three  trials  (approximately  100  hours  of  flying) 
90%  of  pilots  should  progress  to  the  'Extremely  Dangerous' 
category. 


Figure  4  -  JOUST  ranking  system. 

Joe  in  a  1  V  1  scenario  is  at  the  top  of  the  'Extremely 
Dangerous’  category  and  in  a  bigger  scenario  such  as  4  v  4  he 
is  towards  the  bottom  of  this  category.  The  reason  for  the 
difference  is  that  4  v  4  is  a  more  confusing,  tactically 
challenging  scenario  than  1  v  1,  which  is  dominated  by 
aircraft/weapon  performance.  Considering  Joe's  strengths  and 
weaknesses,  his  greatest  strength  is  in  energy  management 
and  accurate  flying,  which  taken  in  isolation  would  put  Joe 
well  into  the  'Elite'  categoiy. 

4  THE  DEVELOPMENT  OF  RITA 


A  natural  development  of  the  work  carried  out  on  Joe  was  to 
try  to  develop  Joe  into  some  form  of  tactical  adviser  for  a 
pilot,  A  prototype  system  was  developed  called  RITA  (Real- 
Time  Intelligent  Tactical  Adviser)  that  used  a  standard 
manned  JOUST  station  and  software  that  incorporated  a 
modified  version  of  Joe's  tactics  (see  Figure  5).  Interaction 
with  RITA  is  accomplished  via  Direct  Voice  Input  (DVI)  and 
Direct  Voice  Output  (DVO).  The  DVI  equipment  was 
procured  from  the  DERA  Speech  Research  Unit  based  at 
Malvern  and  is  called  an  AURIX.  The  AURIX  has  the 
advantage  that  it  is  speaker  independent  and  has  a  very  user 
friendly  development  environment,  which  makes  it  ideal  for 
rapid  prototyping.  In  practice  the  AURIX  has  achieved  a 
recognition  success  rate  in  excess  of  90%  and  has 
demonstrated  that  it  will  tolerate  many  different  user  accents. 

The  features  currently  incorporated  into  RITA  are  - 


There  are  probably  people  reading  this  paper  who  disagree 
with  the  preceding  comments  and  think  this  approach  has 
very  little  going  for  it  compared  to  more  modem  Artificial 
Intelligence  techniques.  However  at  the  end  of  the  day  there  is 


a.  Different  stages  of  auto-pilot  control. 

b.  Sensor  management. 

c.  Targeting  advice. 

d.  Energy  management  advice. 


Ill 


e.  Steering  advice. 

The  major  problem  at  the  moment  is  in  the  quality  of  Joe's 
tactics  and  this  obviously  hinders  the  type  of  advice  that  can 
be  given  to  the  pilot.  Remembering  that  Joe's  strengths  lie  in 
the  areas  of  accurate  flying  and  energy  management  it  is  not 
surprising  that  the  most  useful  RITA  features  are  the  different 
forms  of  auto-pilot  control. 

The  first  form  of  auto-pilot  is  a  conventional  heading/pitch 
auto-pilot  but  with  the  auto-pilot  commands  such  as  desired 
heading  being  input  via  the  DVI  system.  The  next  form  of 
auto-pilot  flies  combat  manoeuvres  such  as  a  F-pol 
manoeuvre  after  missile  launch.  During  the  F-pol  RITA  will 
fly  the  aircraft  to  keep  the  target  within  a  few  degrees  of  the 
edge  of  the  sensor  coverage,  whilst  at  the  same  time  adjusting 
speed  and  height  for  tactical  reasons.  When  the  missile  no 
longer  requires  sensor  support  RITA  will  inform  the  pilot  via 
DVO  that  the  F-pol  manoeuvre  is  finished  and  then  hand 
control  of  the  aircraft  back  to  the  pilot.  The  last  form  of  auto¬ 
pilot  performs  all  the  flying  (engaging,  f-poling,  defensive 
flying,  re-engaging  etc.)  leaving  the  pilot  free  to  concentrate 
on  the  more  complicated  tactical  aspects  of  the  combat. 

All  the  auto-pilot  functions  can  be  modified  in  terms  of  how 
much  'g'  they  pull,  by  the  use  of  DVI  commands.  Combat 
manoeuvres  can  be  customised  via  the  DVI,  such  as  in  the  F- 
pol  manoeuvre  the  direction  of  the  F-pol  can  be  defined, 
overriding  the  direction  suggested  by  RITA.  When  the  auto¬ 
pilot  performs  all  the  flying  the  pilot  has  control  over  items 
such  as  which  target  is  engaged. 

An  aircraft  that  effectively  flies  itself  in  combat  has  met  with 
mixed  responses  from  aircrew  interviewed.  Surprisingly, 
pilots  are  happier  about  the  prospect  than  are  the  backseat 
navigators.  This  is  probably  because  the  navigators  have  more 
experience  of  getting  thrown  about  when  the  aircraft  performs 
unpredicted  manoeuvres. 

This  all  leads  to  an  interesting  conclusion.  The  current  air 
defence  fighter  in  the  RAF  is  the  two  seat  Tornado  F3  and  the 
next  fighter  will  be  the  single  seat  Eurofighter.  The  work  on 
tactical  advisory  systems  was  started  because  of  a  need  to 
help  the  single  seat  pilot.  The  perceived  view  of  a 
computerised  adviser  is  to  replace  the  navigator  in  the  back  by 
some  form  of  'R2D2'.  However  given  the  strengths  of 
computers  compared  to  the  intelligence  of  man,  if  you  really 
want  to  improve  the  combat  effectiveness  of  a  pilot  in  BVR 
air  combat  it  is  better  to  put  'R2D2'  in  the  front  seat  carrying 
out  the  flying  and  have  the  human  in  the  back  seat  planning 
the  tactics  and  gathering  situational  awareness. 

5  CONCLUSIONS 

The  approach  to  producing  a  digital  pilot  for  air  combat  has 
proven  very  successful  and  Joe  is  continuing  to  be  developed 
such  that  he  can  address  a  wide  range  of  operational  analysis 
issues.  A  new  role  for  Joe  is  to  be  part  of  the  research  being 
carried  out  in  the  emerging  field  of  combat  Uninhabited  Air 
Vehicles  (UAVs). 

The  development  of  RITA  has  proven  less  successful.  The 
major  problem  with  this  work  is  in  the  quality  of  Joe's  tactics 


compared  to  an  average  pilot  and  this  obviously  hinders  the 
type  of  advice  that  can  be  given  to  a  pilot.  If  the  pilot  feels  he 
can  not  rely  on  Joe  to  make  good  decisions  and  has  to 
continually  monitor  Joe's  actions  then  this  could  actually 
increase  his  workload.  If  in  the  fullness  of  time  Joe  becomes 
very  adept  at  making  tactical  decisions,  then  you  are  faced 
with  the  philosophical  question;  do  we  actually  need  a  pilot  in 
the  cockpit? 

At  present  it  seems  the  best  way  to  improve  the  combat 
effectiveness  of  a  single  seat  pilot  is  to  provide  him  aids  to  his 
own  tactical  decision  making  process,  rather  than  to  provide 
him  with  direct  tactical  advice  which  at  best  will  be  mediocre. 
An  example  of  a  current  aid  in  the  cockpit  is  the  display  of  a 
missile  launch  success  zone  (LSZ).  The  LSZ  tells  the  pilot  if 
his  opponent  is  within  range  and  where  the  range  is  with 
respect  to  an  'edge  of  the  envelope’  or  a  'heart  of  the  envelope' 
shot.  The  LSZ  does  not  tell  the  pilot  when  to  shoot,  that  is  a 
decision  he  makes  by  balancing  the  risk  level  to  himself  to 
how  badly  he  wishes  to  shoot  down  his  opponent.  Other  work 
(ref  1  &  2)  has  shown  that  pilots  doubt  the  ability  of  any 
computer  algorithm  to  correctly  account  for  all  the  tactical 
factors  they  consider  during  combat.  Further  work  should 
concentrate  on  developing  new  aids  which  will  help  the  pilot 
in  judging  the  relative  merits  of  a  particular  course  of  action 
he  selects. 

An  alternative  method  of  improving  the  pilots  effectiveness  is 
to  reduce  his  workload,  so  allowing  him  more  time  to  devote 
to  tactical  thinking.  Allowing  RITA  to  fly  the  aircraft  in 
combat  manoeuvres  is  an  example  of  this.  However  this 
example  in  particular  is  likely  to  meet  with  resistance  from 
aircrew.  In  general  the  controls  to  all  the  systems  should  be 
de-coupled  such  that  the  pilot  always  issues  high  order 
commands  and  lets  the  systems  work  out  the  best  way  to 
achieve  the  results.  Existing  fly-by-wire  aircraft  already  do 
this  in  that  rather  than  use  the  control  stick  to  move  the 
elevator  directly,  the  control  stick  passes  a  demand  to  the 
flight  control  computer  which  then  moves  the  elevator, 
canards,  flaps,  slats  etc.  appropriately. 

Work  is  continuing  at  Famborough  and  the  major  challenge  is 
coming  up  with  good  ideas  that  are  practical  to  implement, 
will  be  useful  to  aircrew  and  will  be  accepted  by  aircrew. 
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Figure  1  -  Typical  JOUST  system  configuration.  Figure  2  -  JOUST  system  cockpit  configuration. 


Figure  3  -  JOUST  system  head  down  instrumentation.  Figure  5  -  RITA  Architecture 
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SUMMARY 

Speech  is  a  natural  form  of  communication  between 
humans.  It  should  come  as  no  surprise  that  it  would 
also  be  the  ideal  form  of  communication  between  a  pilot 
and  an  electronic  crewmember.  High-level  commands 
spoken  by  the  pilot  would  be  interpreted  and  carried  out 
by  the  electronic  crewmember  in  much  the  same  way 
that  a  pilot  would  talk  to  another  crewmember.  The 
realization  of  this  natural  interface  will  depend  on  a 
robust  speech  recognition  capability  to  handle  the 
degraded  speech  conditions  typical  of  the  military 
aircraft  environment-  This  paper  reviews  the  latest 
progress  in  robust  speech  recognition  research  and  its 
potential  application  for  military  aircraft.  Sources  of 
degradation  in  the  speech  signal  will  be  discussed  along 
with  the  techniques  being  explored  to  reduce  their 
effects  on  speech  recognition.  Results  of  recent  flight 
testing  will  also  be  presented  to  provide  a  benchmark  of 
the  performance  of  commercially  available  speech 
systems  in  the  military  environment.  Finally,  remaining 
challenges  to  providing  a  fiilly  capable,  high-accuracy 
speech  interface  to  the  electronic  crewmember  will  be 
discussed. 

1  INTRODUCTION 

Speech  technology  holds  the  promise  of  providing  a 
natural  means  for  human  crewmembers  to  communicate 
with  their  future  electronic  counterparts.  Automatic 
speech  recognition  is  a  rapidly  emerging  human- 
computer  interface  technology  that  will  provide  a  safe 
and  efficient  method  for  handling  the  complex 
information  management  requirements  of  future  fighter 
aircraft.  The  Vehicle-Pilot  Integration  Branch  in 
Wright  Laboratory  (WL)  has  been  actively  investigating 
the  potential  of  this  technology  for  over  twenty  years. 
Flight  test  experiments  conducted  in  the  80’ s  provided 
the  first  opportunity  for  WL  to  assess  recognition 
performance  of  first  generation  airborne  speech 
recognition  systems  (ref.  1,  2).  Results  from  these  early 
flight  test  programs  suggested  that  significant 
improvements  in  the  area  of  robust  speech  recognition 
were  needed  before  an  operational  system  could  be 


fielded  in  military  aircraft. 

Robustness  refers  to  the  ability  of  a  speech  recognition 
system  to  operate  under  adverse  conditions.  In  the 
military  environment  these  adverse  conditions  are 
concentrated  in  two  areas:  noise  and  speech  variability. 
Noise  is  produced  by  the  aircraft  engines,  wind, 
environmental  control  system,  oxygen  mask  breath 
noise,  and  electrical  channel  noise  produced  by 
distortion  in  the  microphone  and  avionics  systems. 
Speech  variability  is  primarily  caused  by  g  forces, 
workload  stress,  fatigue,  and  Lombard  speech. 
Lombard  speech  occurs  when  speakers  attempt  to  make 
themselves  heard  over  the  background  noise.  If  speech 
recognition  is  to  be  a  viable  cockpit  information 
management  technology,  researchers  must  fully 
understand  the  factors  that  degrade  the  speech  signal  in 
the  operational  environment  and  develop  robust 
algorithms  to  compensate  for  them. 

Fortunately,  significant  progress  has  been  made  in 
robust  speech  recognition  since  those  first  flight  test 
experiments  in  the  80’ s.  Improvements  in  digital  signal 
processing  technology  are  resulting  in  relatively 
inexpensive  speech  recognition  systems  that  are 
designed  to  operate  in  noisy  industrial  environments  for 
command-and-control  and  data  entry  applications. 
Also,  the  growing  market  for  computer  telephony 
applications  is  resulting  in  technology  capable  of 
recognizing  speech  over  noisy  telephone  lines.  With 
little  modification,  a  system  designed  for  these 
commercial  applications  could  be  adapted  for  use  in 
military  aircraft. 

This  paper  reviews  the  latest  progress  toward  achieving 
a  robust  speech  recognition  interface  for  military 
aircraft.  Sources  of  degradation  in  the  speech  signal 
will  be  discussed  along  with  the  techniques  being 
explored  to  reduce  their  effects  on  speech  recognition. 
Results  of  two  recent  WL  flight  tests  on  two  NASA 
OV-10  aircraft  will  also  be  summarized  to  provide  a 
performance  benchmark  of  commercially  available 
speech  systems  in  the  airborne  environment.  Finally, 
remaining  challenges  to  providing  a  fully  capable,  high- 
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accuracy  speech  interface  to  an  electronic  crewmember 
will  be  discussed. 

2  SOURCES  OF  COCKPIT  SPEECH 
DEGRADATION 

As  mentioned  above,  there  exists  two  primary  areas  that 
can  degrade  the  speech  signal  in  the  military  aircraft 
environment:  noise  and  speech  variability.  Each  of 
these  sources  along  with  ways  of  compensating  for  them 
is  discussed  below. 

Noise  Sources 

Three  major  noise  sources  that  contribute  to 
degradation  of  the  speech  signal  in  a  cockpit  are 
ambient  background  noise,  channel  noise,  and  speaker 
noise.  Ambient  background  noise  is  produced  by  the 
aircraft  engines,  environmental  control  system,  and  the 
sound  of  air  moving  past  the  aircraft.  Channel  noise 
refers  to  the  distortions  produced  by  the  microphone 
transducer  and  electrical  noise  conducted  in  the  wiring 
to  the  speech  system.  Speaker  noise  refers  to  non¬ 
vocabulary  speech  sounds  such  as  lip  smacks,  breath 
noise  from  an  oxygen  mask,  or  grunting  sounds 
produced  when  a  pilot  undergoes  high-g  maneuvers. 

Three  of  the  most  commonly  used  techniques  for 
reducing  the  effects  of  these  noise  sources  are  training 
in  the  environment,  preprocessing,  and  noise 
cancellation  algorithms.  If  the  noise  source  is  relatively 
consistent  and  steady-state,  as  it  is  for  many  airborne 
environments,  then  having  the  speaker  train  the  system 
in  noise  conditions  that  simulate  the  actual  aircraft 
environment  is  one  method  of  producing  representative 
voice  templates  that  will  more  closely  match  commands 
given  in  flight.  Preprocessing  and  noise  cancellation 
are  preferred  over  training  in  noise,  however,  as  these 
techniques  try  to  reduce  noise  effects  by  modification  of 
the  speech  signal  rather  than  requiring  additional 
training  from  speakers  (ref.  3, 4). 

Oxygen  mask  breath  noise  is  a  unique  problem  that  has 
to  be  dealt  with  for  high-performance  aircraft 
applications.  The  creation  of  breath  noise  models  was 
successfully  used  with  the  ITT  VRS-1290  speech 
system  during  the  second  OV-10  flight  test  experiment 
conducted  by  Wright  Laboratory  and  will  be  discussed 
in  the  next  section.  Training  the  system  with  the  oxygen 
mask  also  incorporates  the  intra- utterance  breath  noise 
as  part  of  the  word  models  and  minimizes  its  impact. 

Other  speech  noises  such  as  lip  smacks,  grunts,  or  out- 
of- vocabulary  utterances  are  more  difficult  to  deal  with. 
One  method  is  to  adjust  the  recognition  threshold  of  a 
system  to  reject  out-of-vocabulary  sounds.  The 


problem  with  doing  this,  however,  is  that  there  is  a 
danger  of  the  system  rejecting  too  much  valid  speech  as 
well.  Another  method  is  to  incorporate  “garbage” 
models  that  recognize  and  reject  speech  noises  and  out- 
of-vocabulary  speech.  This  is  typical  of  word-spotting 
systems  that  can  recognize  keywords  in  a  continuous 
stream  of  speech. 

Speech  Variability 

The  second  major  area  of  speech  degradation  occurs 
when  the  pilot’s  voice  changes  due  to  various  factors 
such  as  g  forces,  workload  stress,  fatigue  and  the 
Lombard  effect.  In  previous  flight  test  experiments, 
flying  up  to  6  g’s  resulted  in  little  degradation  in  speech 
recognition  performance  (ref.  2).  Fortunately,  there  is 
very  little  application  for  speech  recognition  above  6  g’s 
and  with  the  proper  closed-loop  feedback  of  g  level  to 
the  speech  system,  the  vocabulary  and  grammar 
structure  can  be  significantly  limited  to  only  enable 
those  few  tasks  that  a  pilot  may  want  to  access  by  voice. 

The  effect  of  background  noise  alone  on  speech 
recognition  performance  is  not  as  detrimental  as  how 
the  noise  affects  a  speaker’s  response  to  it.  This 
Lombard  effect,  named  after  the  French  physician  who 
first  described  its  characteristics  (ref  5),  results  in 
changes  to  a  speaker’s  voice  such  as  increased  vocal 
effort,  greater  duration  of  words  due  to  an  elongation  of 
vowels,  frequency  shifts,  and  deletion  of  certain  ending 
consonants.  While  this  effect  makes  it  easier  for 
humans  to  communicate  in  noise,  it  can  reduce  speech 
recognition  accuracy  by  as  much  as  25  percent.  The 
best  techniques  for  minimizing  Lombard  speech  are 
providing  good  audio  feedback  in  the  speaker’s  headset, 
minimizing  the  noise  through  the  use  of  active  noise 
reduction,  and  feedback  techniques  that  provide  the 
speaker  with  the  gain  level  the  system  is  receiving  (ref. 
6, 7,  8). 

3  OV-10  SPEECH  RECOGNITION  FLIGHT 

TESTING 

To  assess  the  impact  of  these  various  sources  of  speech 
degradation  on  commercially  available  speech 
recognition  systems,  two  flight  test  experiments  were 
recently  conducted  by  WL  on  two  NASA  Lewis 
Research  Center  OV-10  test  aircraft.  The  objectives  of 
these  experiments  were  1)  measure  live  recognition 
performance  in  several  ground  and  flight  test 
conditions,  including  testing  up  to  4g’s  and  2)  generate 
a  digital  speech  database  for  further  research. 
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Experiment  1  -  ITT  VRS-1290  Evaluation  with  an 
M-162  Boom  Microphone 

Sixteen  subjects,  comprised  of  active  duty  military  and 
NASA  pilots,  participated  in  the  evaluation  of  an  ITT 
VRS-1290  speech  recognition  system  installed  in  a 
ruggedized  IBM-PC.  The  aircraft  used  for  this 
experiment  was  an  OV-lOA  aircraft  operated  by  NASA 
Lewis  Research  Center  in  Cleveland,  OH  (Figure  1). 
This  aircraft  was  a  twin  engine,  two  crew  member, 
tandem  seating  turboprop  aircraft.  The  OV-lOA  was 
capable  of  pulling  up  to  5.5  gs,  but  due  to  equipment 
constraints  the  test  profiles  were  limited  to  1  and  3g 
maneuvers. 


Figure  1 .  NASA  LeRC  OV-1 OA  Test  Aircraft 


Vocabulary/Grammar  Structure 

The  vocabulary  consisted  of  53  words  and  phrases  that 
represent  various  tasks  that  could  be  accomplished  in  a 
military  aircraft.  The  vocabulary  and  grammar  structure 
is  shown  in  Table  1.  The  53  vocabulary  words  and 
phrases  were  combined  to  form  91  test  utterances  to  be 
used  during  ground  and  flight  test  conditions. 
Synonymous  words  such  as  Go-to,  Display,  and  Show 
or  page  and  layer  were  designed  into  the  test  vocabulary 
to  allow  a  more  flexible  interaction  with  the  speech 
system, 

{North/South/East/West}  (0-180)  degrees  (0-59) 
point  (0-9  9)  minutes 

Range  {five/ten/twenty/forty/elght/one-sixty/two-forty} 

Give-me  {TF/TA/TF-TA/ground-map/pencil-beam/ 
weather/beacon} 

Change  Radar-mode  [to]  {TF/TA/TF-TA/ground-map/ 
pencil-beam/weather/beacon} 

{Delete/Modify}  N-R-P  (0-9  9) 

Add-New  N-R-P  {before/after}  (0-9  9) 


{Goto/Display/Show}  {IDS/comm/flight- 
director/radar/flight-plan}  {page/layer} 

Table  1 .  Flight  Test  Vocabulary/Grammar 
Test  Procedures 

Each  subject  began  the  experiment  by  performing 
template  generation  followed  by  a  baseline  performance 
assessment.  Template  generation  involved  the  subjects’ 
speaking  a  number  of  sample  utterances  which  were 
prompted  by  the  ITT  system.  Once  template  generation 
was  completed,  a  recognition  test  followed  which 
consisted  of  reciting  91  utterances  twice  to  collect 
baseline  recognition  data.  All  of  the  laboratory  training 
and  testing  utterances  were  recorded  on  digital  audio 
tape  (DAT)  to  allow  subsequent  testing  on  the  ITT 
system  or  testing  of  a  new  speech  recognition  system. 


The  subsequent  test  sessions  were  conducted  on  the 
aircraft  both  on  the  ground  with  no  engines  running  and 
in  the  air.  During  data  collection,  subjects  sat  in  the 
rear  seat  of  the  OV-lOA  and  were  prompted  with  a 
number  of  utterances  to  speak.  All  prompts  appeared 
on  a  5”  X  7”  monochromatic  liquid  crystal  display  in  the 
instrument  panel  directly  in  front  of  the  subject.  The 
ITT  system  attempted  recognition  after  each  spoken 
phrase  with  the  results  stored  for  later  analysis.  Once 
again,  DAT  recordings  were  made  of  the  entire  data 
collection  session.  After  the  ground  test  was  complete, 
the  subjects  flew  the  flight  test  profile  consisting  of 
three  conditions:  1)  straight  and  level  flight  (IGl),  2)  3g 
flight  (3G),  and  3)  repetition  of  the  Ig  condition  to 
examine  potential  fatigue  effects  (1G2). 

Results 

During  the  first  several  flights,  ITT  word  accuracy  was 
around  55%.  In  the  course  of  investigating  potential 
causes  for  this  performance  degradation,  several 
problems  were  discovered.  These  problems  were 
primarily  audio  related  but  also  had  to  do  with  several 
engineering  parameters  that  controlled  the  ITT  system. 
After  consultation  with  ITT  researchers,  DAT  flight 
test  recordings  were  replayed  into  the  system  on  the 
ground  with  systematic  adjustments  to  the  gain  and 
engineering  parameters.  Recognition  performance  was 
then  obtained  at  greater  than  98%.  Once  this 
performance  optimization  was  accomplished,  live 
performance  was  maintained  at  98%  or  better  across  all 
flight  conditions  with  no  significant  degradation  at  3g’s. 

Due  to  the  audio  and  system  problems  encountered 
during  the  experiment,  only  five  of  the  sixteen  subjects 
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had  valid  real-time  recognition  performance  data  in¬ 
flight.  Four  of  the  sixteen  subjects  experienced 
problems  with  the  DAT  recording  equipment,  resulting 
in  unusable  or  non-existent  audio  data.  Audio 
recordings  were  successfully  collected  for  a  total  of 
twelve  subjects  in  the  study. 

The  data  analyses  were  done  in  two  stages.  The  first 
stage  involved  a  comparison  of  “live”,  in-flight  word 
recognition  performance  with  word  recognition 
performance  obtained  by  playing  the  DAT  recordings 
made  in-flight  into  the  ITT  system  back  in  the 
laboratory.  The  premise  was  that  if  no  significant 
differences  were  found  between  live  vs.  DAT 
performance  on  the  five  subjects  that  flew  with  the 
optimum  configuration,  then  the  remaining  subjects 
with  complete  DAT  audio  could  be  retested  in  the  lab  in 
the  same  way.  Figure  2  shows  the  mean  word 
recognition  performance  for  both  live  and  DAT 
recordings  for  the  five  subjects  who  had  valid  in-flight 
data. 

I  DAT 

□  live 


1G1  3G  1G2 

FLIGHT  CONDITION 

Figure  2.  Mean  word  accuracy  for  live  and  DAT 
testing 
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Lab  Hangar  1G1  3G  1G2 
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Figure  3.  Mean  word  accuracy  for  each  test 
condition. 

Experiment  2  -  ITT  VRS-1290  and  Verbex  VAT31 
Evaluation  with  an  M-169  Oxygen  Mask 
Microphone 

A  second  experiment  was  conducted,  this  time  using  an 
oxygen  mask  with  an  Air  Force  standard  M-169 
microphone  to  examine  the  effects  of  aircraft  noise, 
breath  noise  and  g’s  on  speech  recognition  performance. 
Ten  subjects  participated  in  the  first  stage  of  the 
experiment  which  evaluated  the  ITT  VRS-1290  speech 
recognition  system,  this  time  installed  on  a  NASA  OV- 
lOD  aircraft.  After  the  ITT  testing  was  completed,  a 
Verbex  VAT31  was  installed  and  evaluated  with  six 
subjects  using  the  same  vocabulary  and  grammar 
structure.  Since  different  subjects  were  used  for  both 
systems,  a  direct  comparison  between  the  ITT  and 
Verbex  was  not  performed.  Also,  both  ITT  and  Verbex 
were  consulted  to  ensure  optimum  performance  for  both 
systems. 
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An  Analysis  of  Variance  revealed  no  significant 
differences  in  word  recognition  performance  when 
providing  the  ITT  system  with  both  live  and  digitally 
recorded  audio  signals.  With  no  performance 
differences  found  between  live  and  DAT  audio  signals, 
all  of  the  remaining  analyses  were  done  using  DAT 
audio  tape  as  the  input  to  the  VRS-1290.  This  provided 
complete  recognition  data  for  twelve  subjects.  Figure  3 
shows  the  mean  word  recognition  performance  obtained 
for  each  of  the  test  conditions.  Statistical  analysis 
revealed  no  significant  differences  in  any  of  the  test 
conditions. 


Vocabulary/Grammar  Structure 

A  new  vocabulary  and  grammar  structure  was 
developed  for  this  experiment.  The  vocabulary 
consisted  of  47  words  and  phrases,  some  of  which  were 
used  during  the  second  AFTI/F-16  flight  test  that  was 
performed  over  ten  years  ago  (ref.  2).  The  vocabulary 
and  grammar  structure  is  shown  in  Table  2.  A  total  of 
57  test  utterances  was  developed  to  be  used  during 
ground  and  flight  test  conditions. 

(Uniform/Comm  1)  (2  0  0  0  -  3  9  9  9) 

(Victor/Comm  2)  (1  5  0  0  -  1  9  9  9) 

(Uniform/Comm  1)  (button/channel)  (1-2  0) 
(Vlctor/Comm  2)  (button/channel)  (1-2  0) 

Radar  range  (tenAwenty/forty/eighty) 

Radar  azimuth  (tenAhirty/sixty) 

Radar  (1/2/3/4)  bar 

(HI-TACANALS)  runway  (00-36)  (Left/Right) 
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(Say/Whats-my)  (fuel/inventory) 

(TrackeryTargetIng-pod)  (wide/narrow/cursor-zero) 
(Tracker/Targeting-pod)  (black/white)  hot 
CCIP 

Air-to-air-mode 
Air-to-Grou  nd-mode 
Nav-mode 
Strafe 

Table  2.  OV-1 OD  Flight  T est  Vocabulary 

Test  Procedures 

The  test  procedures  for  both  the  ITT  and  Verbex 
systems  were  repeated  from  the  first  experiment.  Each 
subject  began  the  experiment  by  performing  template 
generation  followed  by  a  baseline  performance 
assessment.  After  that,  a  recognition  test  followed 
which  consisted  of  reciting  57  utterances  twice  to 
collect  baseline  recognition  data.  Once  again,  all  of  the 
training  and  testing  utterances  were  recorded  on  DAT  to 
allow  follow-on  testing. 

The  subsequent  test  sessions  were  conducted  on  the 
aircraft  both  on  the  ground  with  no  engines  running  and 
in  the  air.  After  the  114  utterance  ground  test  was 
complete,  the  subjects  flew  the  flight  test  profile 
consisting  of  three  conditions:  1)  114  utterances  at 
straight  and  level  flight  (IGl),  2)  70  utterances  in  4g 
flight  (4G),  and  3)  114  utterances  repeating  the  Ig 
condition  at  a  higher  engine  throttle  setting  to  induce 
more  noise  for  this  condition.  (1G2). 

Results  -  ITT  Testing 

Due  to  various  data  recording  and  aircraft  problems, 
DAT  audio  was  successfully  recorded  for  only  eight  of 
the  ten  subjects  under  all  test  conditions.  Live 
recognition  performance  was  obtained  for  five  of  these 
eight  subjects.  Figure  4  shows  the  word  accuracy  for 
five  subjects  under  each  test  condition.  Two  factors 
account^  for  the  majority  of  the  recognition  errors, 
lack  of  automatic  gain  control  and  Lombard  effect.  As 
a  result  of  the  first  flight  test,  the  ITT  system  was  used 
without  its  automatic  gain  control  circuitry  enabled. 
This  was  because  the  ITT  system  had  difficulty 
converging  on  the  proper  gain  setting  when  exposed  to 
high  noise.  Also,  when  it  finally  did  settle  on  a 
particular  gain  setting,  it  was  found  that  the  signal  was 
too  strong  to  obtain  accurate  results.  So  fixing  the  gain 
to  a  predetermined  value  was  the  only  way  to  get  the 
ITT  system  to  function  reasonably  well  in  this  second 
flight  test. 


Figure  4.  ITT  word  accuracy  for  five  subjects 

This  became  a  problem,  however,  with  subjects  that  had 
a  pronounced  Lombard  effect,  particularly  in  the  4G 
condition.  During  the  4G  run,  the  noise  level  increased 
by  an  average  of  22  dB  from  the  IGl  condition.  This, 
coupled  with  a  lack  of  good  sidetone  in  the  subjects’ 
helmet  earcups,  resulted  in  some  subjects  almost 
shouting  to  compensate  for  the  increased  noise  level. 
This  explains  why  subject  2’s  word  recognition  results 
at  4G  were  at  72.7%.  Subject  5,  however,  was  very 
accustomed  to  speaking  in  the  OV-10  and  consequently 
was  able  to  maintain  performance  at  95%  or  better  for 
all  conditions.  Average  performance  over  the  five 
subjects  was  97.2%  for  the  two  ground  conditions  and 
92.1%  for  the  three  flight  conditions.  Subsequent 
experiments  are  planned  with  the  DAT  audio  to 
determine  if  gain  normalization  will  improve  ITT 
performance. 

Results  -  Verbex  Testing 

Five  of  the  six  subjects  had  complete  live  data  and  DAT 
audio  for  each  of  the  five  test  conditions.  Figure  5 
shows  the  live  word  recognition  performance  for  each 
of  the  five  subjects.  Subject  5  is  the  same  subject  as 
subject  5  in  the  ITT  test.  Due  to  his  experience  with 
both  the  aircraft  and  the  testing  procedures,  his 
performance  was  the  best  at  100%  under  all  conditions. 
Subject  3  was  a  non-pilot  subject  that  showed  a 
pronounced  Lombard  effect  under  4g’s.  This  explains 
the  performance  degradation  of  86%  in  the  4G 
condition.  Overall,  the  system  achieved  an  average 
word  accuracy  of  99.5%  in  the  ground  conditions  and 
97.3%  in  the  flight  conditions. 


119 


100 


Figure  5.  Verbex  word  accuracy  for  five  subjects 

OV-10  Flight  Test  Conclusions 

The  two  flight  test  programs  summarized  here  provided 
an  excellent  opportunity  to  obtain  practical  experience 
with  the  airborne  evaluation  of  commercially  available 
speech  recognition  systems.  Perhaps  of  greater 
significance  than  the  actual  recognition  results, 
however,  is  the  fact  that  an  extensive  digital  speech 
database  was  recorded  that  will  be  distributed  to  other 
speech  recognition  researchers  to  develop  and  evaluate 
recognition  algorithms  for  the  airborne  environment. 
This  database  will  also  be  used  internally  to  evaluate 
other  candidate  speech  systems  without  going  through 
the  expense  of  additional  flight  testing.  Of  particular 
interest  is  the  evaluation  of  several  speaker  independent 
systems  that  do  not  require  training  prior  to  use. 

4  FUTURE  DIRECTIONS  AND  CHALLENGES 

A  robust  speech  interface  in  the  cockpit  is  fast 
becoming  a  cost  effective  technology  option  for  crew 
systems  designers.  With  digital  signal  processing  speed 
increasing  about  20  percent  each  year,  the  necessary 
horsepower  required  to  provide  high  accuracy,  real-time 
speech  recognition  in  the  military  environment  is 
already  here  for  small  vocabulary,  continuous  speech 
command  and  control  applications.  With  the  latest  push 
to  adopt  commercial  technology  for  military  use,  the  Air 
Force  can  leverage  a  tremendous  investment  by 
commercial  developers  working  on  robust  speech 
interfaces  to  automobile  systems,  cellular  telephone 
dialing,  information  kiosks,  personal  digital  assistants, 
etc.  While  none  of  these  environments  can  fully 
compare  with  the  operational  fighter  environment, 
technology  gains  made  in  the  private  sector  will  have 
direct  application  to  the  military. 

Several  new  approaches  to  improving  robust  speech 
recognition  are  also  showing  a  lot  of  promise  in  the 


laboratory.  Researchers  are  exploiting  the  use  of  neural 
networks  for  improved  pattern  recognition  and  auditory 
modeling  techniques  that  mimic  the  excellent  noise 
filtering  characteristics  of  the  human  auditory  system 
(ref  9, 10). 

Once  this  robust  speech  processing  capability  is 
available,  the  remaining  challenges  lie  in  the  application 
designer  developing  a  natural,  intuitive  interface 
between  the  pilot  and  the  electronic  crewmember. 
Speech  understanding  systems  that  are  able  to  interpret 
meaning  from  spontaneous  conversational  speech  input 
are  still  in  their  infancy.  Fortunately,  fighter  pilots 
have  little  need  for  verbose  discourse  with  their  aircraft 
and  would  rather  communicate  in  very  short, 
unambiguous  commands.  No  other  human-computer 
interface  technology  has  the  potential  for  providing  as 
rapid  and  efficient  an  interface,  allowing  the  pilot  to 
respond  to  mission  events  at  a  higher  level  of  control 
and  provide  the  timely  decision  support  needed  to 
return  home  safely. 
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1.  SUMMARY 

The  concept  of  an  electronic  crew-member  (EC)  and 
its  application  in  US  Air  Force  aircraft  has  been 
considered  for  many  years.  The  details  of 
implementing  this  concept  are  divided  among  various 
groups  of  people.  One  such  group  comprises  the 
artificial  intelligence  programmers  who  struggle  with 
the  transition  of  knowledge  into  software  code. 
Another  group,  pilots  who  would  provide  the 
knowledge  base  for  the  EC,  are  concerned  with 
always  maintaining  “command”  of  the  aircraft. 
Finally,  the  cockpit  design  group  struggles  with 
designing  an  optimum  interface  between  the  pilot  and 
the  EC.  It  is  from  this  third  viewpoint  that  the 
methods  by  which  the  pilot  and  the  EC  might 
potentially  communicate  with  one  another  will  be 
discussed. 

This  paper  examines  the  current  method  of 
facilitating  communication  between  the  pilot  and 
computers  on  board  the  aircraft,  as  well  as  some 
futuristic  ideas  for  communication.  One  such 
method,  which  is  the  focus  of  this  paper,  is  speech 
recognition  and  feedback.  The  integration  of  this 
technology  into  a  fighter  aircraft  application,  as  well 
as  lessons  learned  in  laboratory  simulation  will  be 
discussed. 

2.  INTRODUCTION 

Pushing  buttons  or  switches  in  the  cockpit  is  the 
current  method  by  which  the  pilot  and  the  computers 
on-board  the  aircraft  communicate.  Whether  the 
switches  are  bezel-mounted  around  a  Multi-Function 
Display  (MFD)  or  on  the  throttle  and  stick,  pilots 
must  master  location  and  control  logic  sequence  to 
make  changes  to  the  cockpit  setup.  Having  command 
of  on-board  systems  via  the  stick  and  throttle  is  a  very 
attractive  concept  because  the  pilot  can  maintain 
control  of  the  aircraft  while  still  exercising  various 
options  in  the  cockpit.  This  concept,  referred  to  as 


Hands  On  Throttle  And  Stick  (HOT AS)  operation,  is 
present  in  most  of  the  US  Air  Force’s  fighter  aircraft 
(Ref  1). 

With  continuing  exponential  growth  in  computer 
technology,  a  corresponding  growth  in  cockpit 
subsystem  functionality  is  also  expected  well  into  the 
future.  Aircraft  computers  will  be  able  to  process 
more  on-board  information  as  well  as  accept  vast 
amounts  of  data  from  off-board  sources  such  as 
Airborne  Warning  And  Control  Systems  (AW ACS), 
satellites,  ground  controllers,  a  wingman,  etc.  As  the 
amount  of  data  coming  into  the  cockpit  increases, 
stick  and  throttle  designers  will  soon  run  out  of  space 
on  these  controls  to  integrate  more  switches.  With 
this  potential  increase  in  the  amount  of  data,  a  new 
interface  that  complements  the  HOT  AS  concept  will 
be  needed  to  facilitate  operation  of  future  aircraft. 

3.  NEW  EC  INTERFACE  CONCEPTS 
The  use  of  a  touch  sensitive  overlay  on  display 
surfaces  has  been  investigated  as  an  alternate  way  of 
communicating  with  on-board  computers.  Touch 
control  is  a  very  intuitive  method  of  operation  -  little 
training  is  needed  for  one  to  be  able  to  master  this 
type  of  control  in  a  short  period  of  time.  Touch 
control  has  been  tested  in  a  number  of  applications. 
Liggett,  Kustra,  and  Reising  (Ref  2)  showed  that  for 
a  target  designation  task,  touch  control  provided 
faster  response  times  than  the  target  designation 
control  on  the  throttle  (standard  method  of 
designating  targets  in  fighter  aircraft).  However, 
other  research  has  shown  that  pilots  expressed 
concern  about  the  use  of  touch  control  in  terms  of 
accidental  touches,  response  time,  and  loss  of  tactile 
feedback  (Ref  3).  Additional  concerns  include  the 
potential  for  visual  parallax,  and  more  importantly, 
the  use  of  such  a  system  when  pulling  Gs.  Obviously, 
if  fighter  pilots  are  in  a  7  G  turn,  they  won’t  be  lifting 
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their  hand  to  touch  their  MFD  because  the  g-loading 
will  be  too  extreme  to  do  so. 

Touch  control  is  attractive  because  it  is  so  intuitive. 
Even  more  intuitive  might  simply  be  looking  at  items 
of  interest  and  communicating  with  the  cockpit  in  this 
manner.  Two  types  of  controls  that  fall  into  this 
category  are  eye  and  head  tracking.  Eye  tracking 
consists  of  equipment  that  reads  the  pilot’s  retinal 
location  to  determine  where  the  eye  is  focusing.  This 
can  be  achieved  with  an  infrared  corneal  reflective 
system  (Ref.  4).  Head  tracking  consists  typically  of  a 
magnetic  or  ultrasonic  unit  that  is  attached  to  the 
pilot’s  helmet  and  sends  information  to  the  computer 
as  to  the  position  of  the  head. 

If  we  take  the  intuitive  control  logic  one  step  further, 
instead  of  looking  at  an  area  of  interest,  one  might 
just  think  about  it.  Thought  control,  achieved  from 
electrical  impulses  taken  from  the  brain’s  neurons 
(Ref.  5),  is  being  investigated  as  a  possible  control 
method  for  future  applications.  This  type  of  control 
mechanization,  however,  is  years  away  from  being 
incorporated  into  a  cockpit. 

3.1  A  Near  Term  Solution 

One  type  of  control  and  display  method  that  has  been 
getting  much  attention  lately  is  voice  interaction. 
Voice  interaction  usually  consists  of  speech 
recognition  paired  with  audible  feedback,  although 
visual  feedback  is  often  provided  after  receipt  of  a 
speech  command.  The  following  sections  discuss 
how  the  technology  might  be  used,  what  its 
advantages  are,  and  how  well  it  has  been  performing 
in  recent  flight  tests. 

In  the  normal  course  of  flight,  pilots  wear  a  headset  or 
oxygen  mask  and  microphone  to  communicate  with 
other  crewmembers,  air  traffic  control,  a  wingman, 
AW  ACS  operators  and  others.  Communicating  with 
the  computer  via  a  voice  control  could  be  just  as 
natural  as  talking  to  any  other  crewmember.  Also, 
speech  recognition  and  feedback  allows  pilots  to 
maintain  control  of  the  aircraft  by  keeping  their  hands 
on  the  stick  and  throttle  while  communicating  with 
the  aircraft. 

Controlling  aircraft  systems,  receiving  information 
from  on-board  computer  systems,  and  managing  off- 
board  data  can  all  be  done  much  more  safely  and 
efficiently  using  voice  control.  For  example,  one 
voice  command  can  replace  a  number  of  button 
pushes  to  access  layered  pages  on  an  MFD,  a  task 
that  currently  increases  the  pilot’s  manual  and  visual 
task  load,  and  decreases  “head-up”  time.  Anything 


that  can  be  accomplished  by  a  button  push  can  also  be 
done  by  the  voice  system.  Other  potential 

applications  include  asking  the  computer  to  display  or 
remove  information  on  the  instrument  panel, 
configuring  the  cockpit  displays  for  a  specific  mode 
of  flight,  obtaining  a  verbal  message  of  the  amount  of 
fuel  on  board,  and  receiving  verbal  messages 
regarding  distances  to  targets.  The  voice  system 
would  be  a  natural  interface  to  the  mission  computer 
to  achieve  data  entry  and  data  retrieval. 

The  advantage  of  this  type  of  control  is  that  it  allows 
pilots  to  use  the  modality  of  speech,  which  is  in  many 
instances  not  as  overworked  as  are  the  visual  and 
manual  modalities.  In  addition,  speech  recognition 
systems  allow  for  increased  eyes-out  operation.  This 
can  result  in  improved  situational  awareness, 
especially  for  the  single-seat  fighter  pilot. 

One  of  the  biggest  challenges  in  voice  control  is 
ensuring  the  system’s  robustness  in  the  dynamic 
aircraft  environment.  Early  flight  testing  of  voice 
control  systems  (Ref.  6)  showed  a  lack  of  technology 
maturity,  requiring  the  pilot  to  speak  slowly  and 
artificially  to  the  system.  Flight  tests  in  rotary  wing 
aircraft  a  few  years  later  (Refs,  7  and  8)  began  to 
demonstrate  the  emergence  of  continuous  speech 
recognition  with  higher  recognition  accuracy,  but 
showed  other  problems  such  as  high  levels  of  aircraft 
noise  being  recognized  as  speech  (referred  to  as  false 
alarms).  Recent  flight  testing,  however,  has 
demonstrated  that  accuracies  as  high  as  99%  are 
attainable  with  commercial  speech  systems  at  noise 
levels  as  high  as  120  dBA.  (Refs.  9  and  10), 

These  recent  successful  flight  test  results  established 
the  viability  of  speech  recognition  systems  in  an 
actual  aircraft  environment.  Simply  attaining 
acceptable  word  recognition  accuracy  with  any 
benign  vocabulary,  though,  does  little  to  reduce  pilot 
workload  and  increase  mission  effectiveness  and 
survivability.  Considerable  effort  must  be  dedicated, 
before  fielding  a  system,  to  determine  the  mission 
requirements  for  speech  recognition,  to  develop  a 
vocabulary  and  syntax  that  will  support  these  needs, 
and  to  optimize  the  vocabulary  for  increased  accuracy 
and  mission  effectiveness, 

3.2  Developing  a  Fighter  Aircraft  Application 

The  process  of  developing  a  vocabulary  and  syntax 
for  a  fighter  aircraft  application  is  a  rather  time- 
consuming  one  involving  a  number  of  iterative 
phases,  as  outlined  in  Figure  1.  All  phases  might  be 
accomplished  two  or  more  times,  with  each  resulting 
in  a  further  refinement  of  the  vocabulary  and  syntax. 
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Figure  1.  The  Syntax  Development  Process 

The  first  step  entails  an  analysis  of  the  tasks  to  be 
performed  in  order  to  identify  instances  when  the 
pilot’s  hands  and  eyes  might  be  busy.  It  is  during 
these  instances  that  using  the  speech  channel  would 
be  ideal.  Once  the  candidate  mission  tasks  are 
identified,  a  syntax  design  team  generates  a  list  of 
candidate  vocabulary  words  or  phrases  that 
accomplishes  those  tasks.  The  design  team  may 
consist  of  software  programmers,  human  factors 
engineers,  and  pilots.  The  team  combines  candidate 
words  to  form  a  syntax  to  be  used  by  a  speech 
recognition  system.  During  the  later  stages  of  the 
development  process,  the  selection  of  a  particular 
speech  recognition  system  might  be  based  upon  the 
attributes  of  the  chosen  vocabulary.  For  example, 
some  systems  are  known  to  be  better  at  continuous 
digit  recognition.  Other  systems  might  excel  at 
filtering  background  noise. 

Once  a  preliminary  vocabulary  and  syntax  application 
is  developed,  it  should  be  taken  to  the  user 
community  to  demonstrate  its  potential  usefulness. 
Feedback  from  the  user  community  refines  the 
terminology  to  adapt  local  jargon. 

The  refined  syntax  is  then  taken  to  the  laboratory  for 
a  test  of  robustness.  Ideally,  members  of  the  user 
community  serve  as  subjects  in  the  first  tests  of  the 
vocabulary,  but  because  of  time,  budget,  and  other 
constraints,  volunteers  from  available  subject  pools 
are  more  commonly  used.  One  or  more  speech 
recognition  systems  may  be  tested  to  determine  the 
best  one  to  use  for  the  application  being  tested. 

After  laboratory  testing,  refinements  are  again  made 
to  the  system  in  preparation  for  a  more  formal  test  in 
a  simulated  mission  scenario.  These  tests  involve 
members  of  the  fighter  pilot  community.  Simulated 
missions  are  flown  in  an  attempt  to  evaluate  the 


recognition  system’s  ability  to  accurately  carry  out 
the  pilot’s  command.  In  addition,  both  subjective  and 
objective  data  are  collected  to  measure  the 
effectiveness  of  using  speech  recognition  to  perform 
the  mission  tasks.  Other  ground-based  simulations 
might  be  conducted  to  evaluate  the  effects  of  other 
environmental  variables,  such  as  noise,  on  the 
recognition  system.  Once  ground-based  simulation  is 
completed,  the  proposed  system  is  taken  to  flight  test, 
where  other  environmental  variables  such  as  Gs, 
vibration,  stress,  etc.  are  present.  The  results  of  flight 
test  determine  if  these  variables  significantly  affect 
recognition  performance. 

The  Vehicle-Pilot  Integration  Branch  within  the  US 
Air  Force  Wright  Laboratory  has  recently  completed 
much  of  the  work  needed  to  accomplish  a  high 
fidelity  flight  test  of  a  speech  recognition  system  in  an 
F-16  research  aircraft.  A  candidate  vocabulary  has 
been  developed,  laboratory  tests  were  conducted 
using  the  recognizer  to  be  installed  in  the  aircraft,  and 
a  first  simulation  effort  has  been  completed  to  solicit 
ideas  and  comments  from  the  fighter  pilot 
community.  The  following  sections  highlight  the 
findings  of  the  laboratory  and  simulation  tests. 

3.2.1  Laboratory  Test  of  a  Candidate  Vocabulary 

A  representative  fighter  aircraft  vocabulary  was 
developed  and  implemented  on  a  Verbex  speaker- 
dependent  speech  recognition  system  to  determine  the 
effects  of  multiple  testing  sessions  on  recognition 
performance.  It  was  hypothesized  that  by  using  the 
system  over  a  period  of  5  days,  subjects  would 
become  increasingly  comfortable  and  consistent  in 
their  use  of  the  system,  resulting  in  better  recognition 
accuracy. 

Twelve  male  Air  Force  military  and  civilian 
volunteers  served  as  subjects  for  the  laboratory  test. 
Approximately  half  of  the  subjects  had  participated  in 
at  least  one  prior  experiment  involving  speech 
recognition,  although  the  average  experience  with  this 
technology  was  less  than  two  hours. 

The  vocabulary  developed  for  this  effort  consisted  of 
90  words  (Table  1)  which  describe  tasks  performed 
by  fighter  pilots  during  a  typical  air-to-ground 
weapon  delivery  task.  The  words  were  combined  to 
form  91  unique  phrases  used  to  prompt  the  subjects 
during  the  test. 

Subjects  were  first  briefed  on  the  nature  of  the  study 
and  then  seated  in  front  of  a  computer  monitor  to 
begin  the  task  of  “training”  the  recognition  system  to 
recognize  their  voice.  This  template  training  session, 
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which  is  required  of  all  speaker  dependent  systems, 
typically  lasted  20  minutes.  Once  template  training 
was  completed,  the  first  of  5  data  collection  trials  of 
the  test  was  completed.  Subjects  were  visually 
prompted  via  the  computer  monitor  to  speak  each  test 
prompt,  presented  one  at  a  time  in  random  order.  The 
subject’s  task  was  to  simply  speak  the  phrase  shown 
on  the  computer  screen  while  the  speech  system 
attempted  recognition  of  the  utterance.  While  only 
91  unique  phrases  were  developed  for  this  test,  a  total 
of  227  phrases  were  presented  during  each  trial. 
Many  of  the  unique  phrases  were  repeated  in  an  effort 
to  collect  at  least  5  utterances  of  each  vocabulary 
word. 

The  test  trial  lasted  approximately  20  minutes.  On 
the  three  consecutive  days  immediately  following  the 
training  and  first  trial  session  day,  subjects  were 
required  to  come  back  to  complete  another  trial  of 
speaking  227  test  prompts.  The  fifth  and  last  trial 
was  run  exactly  10  days  after  completing  the  first 


trial.  Results  were  scored  for  word  recognition 
accuracy 

Adjusted  Overall  Accuracy  (AO A)  (Ref.  11)  was 
computed  by  trial,  as  shown  in  Figure  2.  In  general, 
word  recognition  accuracy  for  this  vocabulary  / 
recognizer  combination  were  on  the  order  of  99%, 
which  was  considered  to  be  acceptable  for  the  rather 
“sterile”  laboratory  conditions  of  low  noise,  low 
stress,  and  high  signal-to-noise  ratio  of  the  audio 
signal  to  the  system.  There  were  no  statistical 
differences  in  recognition  accuracy  among  the 
individual  trials,  indicating  that  subjects  neither 
gained  nor  lost  recognition  accuracy  over  time. 


Figure  2.  Laboratory  Test  Results 

The  ability  of  subjects  to  train  the  recognition  system 
in  less  than  30  minutes,  attain  high  recognition 
accuracy  with  the  selected  vocabulary  /  syntax,  and 
maintain  accuracy  over  a  period  of  days  proved  the 
viability  of  the  system  to  be  used  in  the  more  costly 
and  time-consuming  aircraft  simulation  tests  to 
follow. 

3.2.2  Simulation  Test  of  the  Candidate 
Vocabulary 

Following  the  initial  laboratory  test  of  the  candidate 
vocabulary,  an  aircraft  simulation  test  was  conducted, 
during  which  USAF,  USAF  Reserve,  and  Ohio 
National  Guard  subject  pilots  provided  additional 
input  to  the  vocabulary  and  S3mtax  design. 

The  primary  objective  of  the  simulation  was  to  further 
refine  the  vocabulary  and  to  determine  the  potential 
mission  effectiveness  of  using  speech  recognition 
technology  during  a  simulated  air  interdiction 
mission. 
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Twelve  male  current  and  former  military  pilots 
participated  in  the  simulation.  All  pilots  had  at  least 
200  hours  of  experience  in  modem  fighter  aircraft  (A- 
10,F-15,F-16,F-18,F417). 

Each  of  the  subject  pilots  participated  for  six  to  eight 
hours  of  testing  conducted  over  a  two-day  period. 
The  simulation  scenario  required  the  pilots  to  fly 
eight  low-altitude  interdiction  missions,  originally 
planned  to  destroy  an  enemy  tank  column  and  a 
petroleum  /  oil  /  lubricant  (POL)  site.  During  the 
missions,  pilots  were  re-directed  to  destroy  enemy 
aircraft  parked  on  an  airfield. 

Each  of  the  eight  simulated  flights  contained  three 
segments  in  which  the  pilot  was  required  to  perform 
secondary  tasks  in  addition  to  the  primary  task  of 
flying  the  aircraft.  During  the  first  segment,  pilots 
were  required  to  manually  fly  terrain  following  and 
terrain  avoidance  (TFT A)  while  changing  a  radio 
frequency,  changing  the  planned  route,  and  fixing  a 
flight  control  fault.  In  the  second,  or  target  detection 
/  acquisition  segment  of  the  mission,  pilots  needed  to 
create  a  patch  map  using  the  synthetic  aperture  radar. 
In  the  third  and  final  segment,  pilots  were  required  to 
designate  six  targets  and  deliver  the  weapons  in  a 
single  pass.  The  secondary  tasks  were  performed 
using  either  current  manual  means  (push  buttons  and 
switches)  or  by  speech  recognition. 

Although  most  objective  results  are  still  pending, 
preliminary  findings  revealed  a  significant  decrease  in 
the  time  required  to  perform  cockpit  tasks  related  to 
digital  entry  when  using  the  speech  recognition 
system.  Examples  of  these  tasks  include  entering 
latitude  /  longitude  data  to  accomplish  a  mission 
reroute  and  entering  the  coordinates  of  a  known 
reference  in  order  to  update  the  navigation  system,  A 
complete  set  of  objective  simulation  results  will  be 
the  subject  of  a  future  report. 

The  subjective  comments  of  the  pilots  have  also 
provided  valuable  insight  as  to  the  optimized  use  of 
speech  recognition  in  the  cockpit.  Many  of  their 
suggestions,  as  well  as  the  lessons  learned  from  the 
implementation  of  the  speech  recognition  system,  are 
described  below. 

3.3  Lessons  Learned 

During  both  the  laboratory  and  simulation  studies,  a 
number  of  lessons  were  learned  in  the  development  of 
the  vocabulary  and  the  use  of  the  speech  recognition 
system.  The  major  lessons  learned  are  summarized 
below. 


Pay  strict  attention  to  the  engineering  parameters 
that  are  available  for  the  system.  There  are  a  number 
of  engineering  parameters  that  typically  can  be 
manipulated  on  most  speech  recognition  systems. 
These  parameters  control  functions  like  rejection 
threshold,  noise  cancellation  options,  number  of 
training  passes,  minimum  and  maximum  time-out 
values,  etc.  The  parameters  requiring  most  attention 
dealt  with  the  amount  of  silence  allowed  between 
words  in  a  phrase  and  the  amount  of  silence  required 
at  the  end  of  the  phrase  to  signal  an  end  of  phrase 
event.  An  attempt  was  made  to  allow  slower,  more 
meticulous  speakers  additional  time  between  words  in 
a  phrase,  while  still  providing  good  recognition 
performance  for  “fast  speakers”.  Setting  this  value  to 
300  milliseconds  seemed  to  provide  good  recognition 
performance  for  both  groups  of  speakers. 

The  rejection  threshold  was  another  parameter  that 
consumed  a  considerable  amount  of  design  time. 
Setting  the  rejection  threshold  too  low  resulted  in  an 
unusually  high  false  alarm  rate,  whereas  setting  this 
too  high  caused  the  system  to  reject  properly  spoken 
phrases.  This  value  had  to  be  adjusted  after  any 
vocabulary  or  syntax  change. 

One  training  option  that  was  found  to  be  quite  useful 
was  called  the  “integrated  enrollment”  option,  where 
each  word  to  be  trained  during  the  discrete  pass  was 
presented  in  a  sentence.  The  subject  had  an 
opportunity  to  see  the  word  in  context  before 
pronouncing  it,  which  created  a  more  solid  template 
base  for  the  word  during  subsequent  continuous 
speech  training  passes  for  co-articulation  effects. 

Shorten  verbose  phrases.  Pilots  in  the  simulation 
phase  indicated  that  they  wanted  very  short  phrases  or 
words  to  accomplish  tasks.  For  example,  the 
originally  designed  vocabulary  required  the  pilots  to 
say  “Report  System  Status”  when  requesting  some 
information  about  one  of  the  aircraft’s  systems. 
Pilots  suggested  that  they  only  be  required  to  say  the 
word  “Systems”  when  asking  for  any  system 
information.  The  phrase  “Slew  Next  Steerpoint”  was 
likewise  shortened  to  “Slew  Next”.  On  a  related 
topic,  if  the  Attitude  Director  Indicator  (ADI)  could 
only  be  shown  on  the  left  MFD,  then  the  pilots  only 
wanted  to  simply  utter  the  phrase  “ADI”,  instead  of 
having  to  say  “ADI  Left”  in  order  to  show  that 
information  on  the  left  MFD. 

Continue  to  use  HOT  AS  for  time  critical  functions. 
Functions  that  are  extremely  time  critical  are  better 
left  on  the  stick  and  throttle.  Examples  include  chaff 
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and  flare  dispensing,  dogfight  switch,  weapon 
selection,  etc. 

DonH  take  short  cuts  with  the  syntax.  Take  the  time 
to  program  the  syntax  so  that  the  resulting  phrases 
make  logical  sense.  For  example,  when  programming 
a  syntax  for  a  UHF  frequency  change,  only  the  digit 
words  “2”  and  “3”  should  be  allowed  as  the  first 
words  in  the  phrase,  since  all  valid  UHF  frequencies 
start  with  a  2  or  3.  The  resulting  logic  will  be  more 
complicated,  but  recognition  accuracy  will  be  better 
by  limiting  the  number  of  words  the  recognition 
system  must  consider. 

Use  audible  feedback,  but  keep  it  short.  In  general, 
the  pilots  in  the  simulation  liked  the  audible 
responses  from  the  voice  system  when  asking  for 
information,  but  the  following  guidelines  were 
suggested:  1)  All  aural  feedback  should  contain  at 
most  two  words,  and  2)  Aural  feedback  should  be 
provided  only  when  there  is  no  obvious  visual 
change.  For  example,  when  switching  from  the  air-to- 
ground  mode  to  air-to-air  mode,  a  number  of  obvious 
display  changes  take  place,  negating  the  need  for  an 
aural  indication  of  the  change. 

Use  Voice  ^^Macros’^  whenever  practical  Pilots  do 
not  like  to  type  in  information  such  as  communication 
frequencies  and  waypoint  latitudes  and  longitudes. 
They  really  liked  the  alternative  of  saying  the 
frequency  and  having  the  speech  recognition  systems 
type  in  the  frequency  for  them.  An  even  better 
solution  would  be  to  implement  the  already  pre¬ 
planned  frequency  change  (analogous  to  radio 
presets)  with  one  simple  voice  command.  For 
example,  the  UHF  frequency  “two  three  eight  point 
seven  five”  could  be  coded  as  “blue  seven”.  The  pilot 
would  simply  say  “blue  seven”  and  the  UHF  radio 
would  change  to  that  frequency.  Another 
“intelligent”  system  would  allow  a  pilot  to  say  “Select 
tanks”  from  an  automatic  target  recognition  system 
display  instead  of  saying  “Select  targets  one,  two, 
seven,  three,  six”. 

Recognize  the  differences  between  prompted  and 
spontaneous  speech.  Many  laboratory  tests  of 
speech  recognition  systems  require  subjects  to  speak 
a  visually  prompted  word  or  phrase.  When  the 
prompts  are  removed,  the  subject  must  rely  on 
memory  and  experience  to  effectively  operate  the 
system.  With  a  lack  of  sufficient  practice  and 
experience  with  the  system,  subjects’  speech  is 
usually  disfluent,  resulting  in  sub-optimal  recognition 
accuracy.  The  best  speech  recognition  accuracy 
comes  from  well  trained  and  experienced  speakers. 


4.  CONCLUSIONS 

The  cockpit  research  community  has  generally 
recognized  the  potential  benefits  of  using  speech 
recognition  technology  in  aircraft.  Commercial  off- 
the-shelf  speech  recognition  systems  have  proven  to 
be  extremely  reliable  in  their  ability  to  accurately 
recognize  spoken  commands  in  quiet  laboratory  and 
office  environments.  A  number  of  challenges  must  be 
met,  though,  before  systems  can  be  fielded  in 
operational  aircraft.  These  systems  must  be  proven  to 
work  in  a  cockpit  where  noise,  Gs,  and  vibration  all 
affect  the  way  a  pilot  might  speak. 

Mission  specific  analyses  of  cockpit  tasks  must  be 
conducted  to  identify  instances  where  voice  control 
would  save  time  or  keep  the  pilot  from  having  to  look 
down  into  the  cockpit  to  enter  data  or  retrieve 
information.  Interviewing  the  operational  pilot 
community,  inviting  them  to  see  the  technology  in 
simulation,  and  flight  testing  are  all  excellent  ways  of 
identifying  and  refining  potential  applications  of 
speech  recognition  in  the  cockpit.  Much  of  this  work 
is  iterative.  Changes  are  made  to  vocabulary  and 
syntax  based  on  user  feedback,  and  then  re-tested, 
which  provides  more  feedback  and  hopefully  fewer 
changes.  The  laboratory  and  simulation  work 
described  in  this  paper  provided  valuable  insight  into 
the  vocabulary  and  syntax  requirements  for  a  typical 
air  interdiction  mission.  The  lessons  learned  from 
this  research  will  be  carried  into  flight  test  on  the  F- 
16  in  the  fall  of  this  year. 
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Summary: 

The  Electronic  Crewmember  (EC)  will  not  only  be  able  to 
support  intuitive  situation  awareness,  but  will  also  assist  in 
m^ing  correct  mission  decisions  and  responses.  It  can  do 
this  because  it  develops  a  unified  picture  of  emerging 
situations  by  taking  a  holistic  view  of  all  the  information  and 
data  available  in  the  aircraft. 

The  ultimate  responsibility  for  the  aircraft  and  the  mission 
lies  with  the  human  crew  who  must  therefore  retain  overall 
authority  over  the  actions  of  both  mission  avionics  and  the 
EC.  So,  the  human  and  EC  form  a  team  with  common 
objectives  but  differing  individual  roles.  As  with  any  team 
that  must  display  'right-stuff  qualities,  efficient 
communication  and  interaction  between  its  members  is 
essential.  In  the  high-stress  aircraft  cockpit  environment  the 
interaction  between  the  human  and  EC  is  a  particularly 
complex  activity. 

In  order  to  design  efficient  means  of  conveying  information 
between  the  human  and  the  EC,  actual  information  must  be 
considered  together  with  its  attributes  or  meta-characteristics. 
In  this  way,  information  can  be  passed  round  the  system  as  a 
vector  which  also  contains  data  about  how  it  should  be 
interpreted. 

This  paper  discusses  some  of  the  attributes  and  meta- 
charcteristics  which  are  required  to  support  the  intuitive 
presentation  of  information  to  the  crew. 

1.  Rationale  for  the  Electronic  Crewmember  -  a  review 

The  benefits  of  incorporation  of  Electronic  Crewmember 
(EC)  capability  into  the  mission  avionics  of  military  aircraft 
are  well  understood.  Associated  research  programmes 
include  the  MMA  project  (UK)  [1],  Pilot’s  Associate 
(USA)[2],  Copilotte  Electronique  (France)[3]  and 
CASSY(Germany)[4]. 

The  amount  of  information,  as  well  as  the  number  of  options 
for  responses,  required  to  carry  out  military  missions,  i.e.  the 
scenario  demand,  is  increasing  over  time,  whilst  the  man- 
machine  bandwidth  capability  remains  static.  This  is 
particularly  the  case  for  the  foreseeable  future  due  to 
increasing  proliferation  of  information  technology  systems 
emerging  in  the  battlefield-  The  EC  has  been  proposed  as  a 
means  of  organising  and  processing  information,  providing 
decision  support  and  keeping  the  man-machine  protocols 
within  the  workload  capabilities  of  the  crew  with  respect  to 
the  ever-increasing  scenario  demand. 

In  broad  terms,  the  carrying  out  of  a  mission  can  be  broken 
down  into  a  number  of  concurrent  management  activities, 
each  of  which  requires  a  particular  man-machine  protocol. 


These  include: 

*  Aircraft  systems  and  vehicle  management; 

*  Defensive  systems  management; 

*  Attack  systems  management; 

*  Weapons  systems  management; 

*  Communications  management; 

*  Position  &  Time  management; 

*  Sensor  management. 

Each  of  these  management  activities  can  benefit  from  EC 
functions  which  can  offer  enhancements  and  improvements 
to  the: 

*  man-machine  protocol  (crew  workload  management); 

*  crew  awareness  of  emerging  total  situation; 

*  correctness  and  efficiency  of  platform  responses. 

In  order  to  provide  for  these  functions,  the  EC  needs  to 
contain  novel  machine  capabilities  such  as: 

*  data  fusion  and  information  management; 

*  situation  assessment; 

*  generation  of  advice  and  decision  support. 

A  system  to  support  these  capabilities  can  be  built  from 
functional  components  such  as: 

*  sensor  data  fusion; 

*  situation  assessment; 

*  planning; 

*  man-machine  interface. 

The  introduction  of  these  components  gives  the  potential  to 
provide  information  that,  if  correctly  presented  to  the  crew, 
can  enable  them  to  assimilate  emerging  situations  more 
readily,  and  respond  more  effectively.  If  the  crew  interface  to 
these  underlying  functions  is  correctly  designed,  it  will  also 
minimise  workload  associated  with  the  assimilation  of 
information  by  the  crew  and  imply  intuitive  responses. 

In  this  way,  missions  will  continue  to  be  able  to  be  carried 
out  efficiently  by  a  small  number  (1  or  2)  of  crew,  without 
them  being  significantly  affected  by  the  increasing  scenario 
demand. 
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However,  this  can  only  happen  if  the  human-EC  team  display 
'right-stuff*  qualities. 

2.  'Right-stuff*  qualities  and  design  issues 

In  considering  a  system  to  support  human-EC  teamwork  with 
'right-stuff  qualities,  design  goals  to  be  taken  into  account 
must  include  such  issues  as: 

*  How  to  fix  the  level  of  automation  in  the  interface  to 
enable  timely  understanding  whilst  leaving  the  crew 
in  the  loop  and  in  charge; 

*  How  to  ensure  that  the  crew  comprehend  what  is 
happening  and  are  not  just  responding  to  directions 
they  do  not  understand; 

*  How  to  achieve  the  right  level  of  trust  so  that  whilst 
cross-checking  is  encouraged,  it  does  not  take  so 
much  time  as  to  make  the  system  useless; 

*  How  to  establish  what  functions  the  crew  require 
from  an  EC  in  terms  of  support  and  'relationship'; 

...  but  most  importantly: 

*  How  to  design  efficient  human-EC  interface 
protocols. 

The  rest  of  this  paper  discusses  the  last  of  these  in  particular. 

Achievement  of  these  design  goals  will  provide  benefits,  but 
not  without  a  trade  off  with  systematic  costs.  Each  of  them, 
at  least  to  some  extent,  loads  the  requirements  for  the  human- 
EC  interface.  So  in  general,  the  benefits  of  introducing  EC 
functions  into  the  aircraft  must  be  considered  in  relation  to 
human  interface  difficulties  and  adverse  systematic  side 
effects  (or  costs). 

3,  Adverse  systematic  side  effects  of  the  Electronic 
Crewmember 

The  behaviour  of  the  cockpit  instruments  not  only  indicates 
information  provided  by  the  functions  of  the  various  avionic 
systems  of  the  aircraft,  but  also  the  correct  functioning  and 
apparent  quality  of  them.  However,  the  EC  will  form  an 
information  layer  between  the  crew  and  the  avionic  systems, 
whose  correct  functioning  will  no  longer  be  able  to  be 
assumed  simply  by  the  presence  or  absence  of  displayed  data. 

As  a  generator  of  information  from  data,  the  very  presence  of 
the  EC  introduces  new  design  requirements  and 
considerations  for  its  human  interface.  These  include  how  to 
impart  an  understanding  of  attributes  such  as: 

*  Uncertainty; 

*  Probability; 

*  Stability; 


*  Assumptions. 

Effectively,  these  attributes  form  meta-characteristics  and  are 
the  qualitative  glue  which  bind  the  human  and  the  EC  into  a 
tight  and  effective  team  with  'right-stuff  qualities.  The  result 
is  that  each  piece  of  information,  combined  with  its  meta- 
characteristics  forms  an  information  vector. 

Meta-characteristics  contribute  not  only  to  the  opinion  the 
human  has  of  the  system,  but  also  vice  versa.  How  can  the 
machine  know  whether  the  man  is  giving  it  flawed 
information?  A  mission  database  may  be  loaded  at  the 
beginning  of  the  mission  and  may  easily  contain  uncertain 
and  incomplete  knowledge  and  assumptions. 

The  following  examples  indicate  situations  where  these  meta¬ 
characteristics  come  into  play  to  varying  degrees: 

e.g.l:  EC:  "There  might  be  a  SAM  site  over 

there  somewhere";  "Or  is  it  a  tank?";  "Oh  no  there  isn't"; 
"Oh  yes  there  is";  "What  do  you  think?" 

e,g.2:  EC:  'Tm  sure  there  is  a  SAM  site  over 

there  and  I  think  it's  a  SAM-13" 

e,g.3:  EC:  "There  is  a  SAM  site  at  co-ordinates 

(x,y)  and  based  on  its  behaviour,  it  is  very  probably  a 
SAM-13" 


e.g.4:  EC:  "That  area  over  there  is  highly 

threatening  because  of  SAM- 13  presence,  it  may  only  be 
navigated  under  200ft,  and  even  then  with  difficulty. 
Your  route  currently  goes  through  it,  and  mission  success 
has  been  reduced  to  30%  as  a  result.  There  are  less  costly 
routes  possible;  would  you  like  to  see  them?" 

In  all  the  information  presentations  shown,  varying  degrees 
of  side  effect  parameters  are  indicated.  These  side  effect 
parameters  introduce  a  new  set  of  design  challenges  and 
considerations  for  the  crew  interface. 

4.  Matching  system  functionality  to  crew  capabilities  and 
requirements 

Introduction  of  EC  technologies  introduces  new  design 
considerations.  For  an  effective  crew  interface  these  fall  into 
four  major  categories: 

*  The  ability  of  the  system  to  present  the  information; 

*  The  ability  of  the  crew  to  assimilate  the  information; 

*  The  requirement  for  trustworthy^  information; 

*  The  ability  of  the  human  to  question  and  cross-check 
the  validity  and  sources  of  the  information. 


^  These  have  been  extensively  discussed  (e.g.  [5]),  and  form 
measurable  high  level  objectives  or  drivers  for  the  design  of  the 
human-electronic  crewmember  system. 
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^  Trustworthy  in  terms  of  both  reliability  of  underlying  systems  as 
well  as  from  a  human-EC  teambuilding  point  of  view. 


In  turn,  the  quality  of  the  provision  of  these  major  factors 
depends  on  several  issues: 

*  Amount  of  information  present  and  cockpit  location 
constraints; 

*  Reliability:  integrity,  fidelity  and  quality  of 
information; 

*  Stability  of  information; 

*  Completeness  of  information; 

*  Aircrew  trust; 

*  Human  limitations. 


4.1  Amount  of  information  present  and  location 
constraints 

With  the  advent  of  new  avionics  capabilities,  the  amount  of 
information  potentially  available  for  display  in  the  cockpit  is 
ever  increasing.  The  provision  of  the  EC  could  be  seen  as 
contributing  to  this  increase.  In  highly  volatile  and 
dynamically  unfolding  situations  'situational  ballooning'  can 
occur,  where  the  number  of  options  to  be  costed  and  created 
by  the  EC  increases  at  a  higher  rate  than  the  number  of 
scenario  considerations  to  be  taken  into  account. 

At  the  same  time,  however,  the  EC  can  provide  functions  to 
manage  and  prioritise  the  underlying  information  to  ensure 
that  the  information  required  is  presented  for  the  particular 
crew  requirements  for  their  current  tasks.  For  example,  an 
information  manager  may  consider  identified  SAM  positions 
and  types  in  relation  to  the  underlying  terrain,  and  calculate 
'safe'  areas  to  fly.  In  turn,  this  'fused*  information  could  be 
used  to  contribute  to  route  costing. 

Further,  with  the  provision  of  more  generic  graphical 
displays  in  the  cockpit,  capabilities  will  be  introduced  to 
configure  and  re-configure  the  presentation  of  information. 
For  example  in  situations  of  possible  display  failure  or 
malfiinction.The  EC  will  therefore  need  to  have  knowledge 
of  the  cockpit  layout  and  availability  of  display  surface  real- 
estate  to  support  the  management  of  presented  information. 

4.2  Reliability;  Integrity,  Fidelity  and  Quality  of 
Information 

The  EC  needs  to  know,  and  be  able  to  convey,  information  to 
the  crew  about  its  knowledge  of  the  quality  of  the 
information  to  be  presented.  If  the  underlying  systems  from 
which  the  data  originates  are  questionable  or  malfunctioning, 
this  needs  to  be  able  to  be  relayed  to  the  crew.  The  reliability 
of  information  sources  must  be  conveyed. 

The  examples  above  are  all  examples  of  qualitative 
presentation.  In  the  case  of  the  last  example,  the  information 
can  be  presented  graphically  very  economically. 

4.3  Stability  of  Information 

The  dynamics  of  the  emerging  situation  may  greatly  affect 
the  operation  of  the  human-EC  team.  The  noise  present  in 


some  information  may  transfer  itself  to  the  displays.  If  the 
EC  cannot  decide,  as  in  the  first  example,  whether  a  piece  of 
information  is  relevant  or  not,  confusion  may  result. 

In  such  'fuzzy*  situations  the  processing  overhead  or 
workload  can  be  very  high.  Decisions  about  interpretation  of 
the  situation  are  required  of  the  man  machine  team,  as  well  as 
what  to  do  about  it.  In  addition,  the  processing  overhead, 
both  for  the  human  and  the  EC,  associated  with  generating 
options  can  become  prohibitive.  By  working  together,  the 
man  and  machine  should  be  able  to  make  a  better  assessment 
of  the  situation  and  response  options. 

It  is  important  that  'fuzzy'  information  is  not  presented  in 
such  a  way  as  to  engender  a  'switch  off  response  by  the  man. 
Two  types  of  switch  off  response  might  be  considered, 
involving  conscious  and  unconscious  but  unilateral 
decisions  by  the  man.  The  first  might  occur  where  the  crew 
decides  that  the  machine  must  be  malfunctioning  because  of 
the  behaviour  of  the  information  presented.  The  second 
occurs  when  the  information  presented  does  not  fit  the 
situation  perceived  by  the  crew;  a  difficulty  to  do  with  belief 
systems. 

As  far  as  possible,  information  must  be  displayed  in  such  a 
way  as  to  present  self-evident  solutions.  Possibly  the  biggest 
threat  to  decision-making  time  comes  from  workload 
associated  with  instability  and  uncertainty  in  information 
presentation;  dithering  is  potentially  fatal;  dithering  is  not 
'right-stuff. 


4.4  Completeness  and  Uncertainty  of  Information 

The  requirement  for  complete  information  will  be  determined 
by  the  set  of  concerns  currently  being  resolved  by  the  crew, 
including  the  need  for  them  to  monitor  the  emerging  general 
situation  in  anticipation  of  predicted  tasks.  If  the  information 
that  can  be  provided  is  incomplete,  it  may  be  preferable  not 
to  use  it  if  its  presentation  will  lead  to  questions  in  the  crew’ s 
mind  and  possible  incorrect  interpretation  and  extrapolation. 
On  the  other  hand  in  some  cases,  incomplete  information 
may  be  better  than  none. 

e.g.5:  EC:  "I'm  still  getting  intermittent  and 

sketchy  tracks  about  the  possible  SAM  site  at  (x,y)  that  I 
told  you  about  5  minutes  ago." 

It  must  be  remembered,  however,  that  complete  information 
is  an  aim  rather  than  an  achievable  reality;  to  some  extent 
there  will  always  be,  and  always  has  been,  uncertainty. 

The  design  of  the  display  must  include  a  qualitative  way  to 
enable  the  representation  of  uncertain  information.  A  display 
would  then  be  able  to  convey  for  example: 

e.g.6:  EC:  "I  am  ninety  something  percent 

certain  that  the  SAM  site  somewhere  within  a  radius  of 
100  meters  of  (x,y)  is  a  SAM- 13." 

Where  there  is  uncertainty,  mistrust  can  follow. 


4.5  Aircrew  Trust 
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Although  introduced  with  the  best  of  design  intentions, 
increasing  sophistication  in  a  technology  can  mask  functional 
complexity,  for  example  through  degrees  of  automation.  This 
has  often  been  viewed  by  crews  with  apprehension,  though 
they  tend  to  display  a  lack  of  understanding  of  automation, 
rather  than  mistrust[8^].  Further,  the  impression  that 
automation  is  not  sufficiently  capable  or  knowledgeable  may 
be  founded  on  experiences  of  ill-considered  implementations 
of  automated  functions. 

The  human-EC  interface  needs  to  be  designed  with  regard  to 
these  issues,  or  the  ability  for  the  EC  and  human  to  work  as  a 
team  in  difficult  situations  will  be  compromised  from  the 
outset.  Essentially,  the  human  crew  needs  to  be  able  to  see 
the  right  information,  at  the  right  time  for  the  task. 

In  order  to  preserve  situation  awareness  and  minimise  any  ill 
effects  upon  workload,  the  crew  need  to  be  sure  that  any 
information  that  is  presented  in  the  cockpit: 

*  Is  supporting  what  the  EC  is  trying  to  achieve; 

*  Contains  sufficient  data/information  to  enable  the 
crew  to  identify  the  tasks  which  have  to  be 
performed; 

*  Contains  sufficient  information  and  data  to  allow  the 
crew  to  perform  their  tasks. 

It  is  necessary  therefore  that  the  interface  to  the  EC  contains 
sufficient  accurate  information  in  .one  form  or  another  for  the 
crew  to  identify  and  perform  their  tasks. 

Understandably,  aircrew  can  be  suspicious  of  untried 
technology  and  concepts;  their  views  can  be  prejudiced  by 
previous  ‘bad’  experiences.  The  potential  areas  for  mistrust 
can  be  better  understood  by  involving  aircrew  in  the  design 
process  from  the  inception.  It  must,  however,  be  borne  in 
mind  that  where  a  crew  of  today  may  not  trust  a  system  due 
to  its  immaturity  or  lack  of  familiarity,  the  crew  of  tomorrow 
may  have  been  brought  up  with  such  concepts,  systems  and 
technologies. 

Aircrew  trust  needs  to  be  built  through  the  provision  of 
sufficient  information  to  allow  them  to  understand  why  the 
system  is  doing  or  saying  what  it  is.  Mission  support 
information  must  be  clear,  concise  and  easy  to  use.  This  is 
particularly  applicable  to  automation  concepts.  [5]  [6]  [7]  [8] 

e.g.7:  EC:  ’’...the  probability  of  mission  success 

can  be  improved  to  90%  by  fine  tuning  the  current  route 
plan.  1  have  6  alternative  options  currently  calculated." 


4.6  Human  Limitations 

Basic  human  capabilities  will  not  change  in  the  future,  but 
what  can  change  is  how  those  capabilities  are  utilised.  This 
will  require  careful  consideration  of  the  benefits  of  the 
technology  to  support  the  crew  of  a  manned  aircraft  and  not 
replace  them. 


^  Refer  to  OPACITY. 


A  key  factor  for  the  crew  interface  system  is  to  design  an 
information  system  which  takes  account  of  the  attributes  of 
both  the  human  and  the  machine. 

It  is  very  important  that  an  equivalent  or  at  least  very  similar 
situation  is  perceived  by  both  the  machine  and  the  man;  their 
individual  perceptions  of  the  situation  should  be  coherent,  so 
that  they  have  a  common  view  of  the  situation  on  which  to 
base  decisions. 

For  example,  difficulties  arise  due  to  the  ability  of  the  human 
to  build  a  mental  model  of  a  situation,  which  can  bias 
judgements  about  an  emerging  situation  to  such  a  degree  as 
to  completely  block  even  important  changes.  [9]  A  glass  half¬ 
full  presents  one  positive  image,  but  a  glass  half-empty 
presents  the  same  information  but  in  a  negative  sense. 

e.g.8:  EC:  "...the  current  plan  has  a  70%  chance 

of  success." 

...or... 

e.g.9:  EC:  "...the  current  plan  has  a  30%  chance 

of  failure." 


5.  Conclusions:  *Right-stuff'  qualities  and  design 
challenges  for  the  human-EC  interface 

The  human-EC  system  must  display  'right-stuff  qualities,  so 
it  is  essential  that  EC  cockpit  interface  is  designed  to  make 
the  EC  a  team  member  rather  than  simply  a  provider  of 
information. 

The  EC  will  have  to  be  designed  and  built  with  respect  to 
human  capabilities.  It  is  important  that  it  is  aware  of  its  own 
shortcomings,  and  conveys  them  to  the  human  crew.  The 
converse  also  applies:  the  crew  must  also  be  aware  of  their 
own  shortcomings,  particularly  with  regard  to  bias,  and 
convey  them  to  the  EC.  Of  course,  this  has  to  be  achieved 
with  respect  to  total  system  (man  and  machine)  workload. 

The  human-EC  system  must  be  designed  from  the  outside  (or 
human  side)  inwards,  with  respect  to,  rather  than  simply  in 
the  basis  of,  its  underlying  functions. 

Requirements  capture  for  the  functions  and  capabilities  of  the 
EC,  however,  presents  possibly  the  greatest  design  challenge. 
For  example,  this  paper  has  considered  in  particular  the  four 
attributes  Uncertainty,  Probability,  Stability  &  Assumptions. 
Are  there  others?  Is  'Appropriateness  to  Mission’  a  possible 
contender? 

The  crew-EC  interface  must  therefore  be  able  to  present  large 
amounts  of  information  in  a  form  that  enables  the  crew  to 
maintain  their  awareness  of  the  dynamically  emerging 
situation  by: 

*  Understanding  what  the  interface  is  telling  them; 

*  Providing  sufficient  information  to  enable  them  to 
identify  and  perform  their  task; 

*  Identifying  any  uncertainty  in  the  data; 

*  Understand  what  the  system  is  recommending  and 
why; 
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*  Allowing  the  crew  to  use  their  abilities  to  best 
advantage  while  supporting  them  in  those  tasks 
which  they  are  less  capable  of  performing. 

Ways  need  to  be  found  to  present  information  with  respect  to 
its  meta-characteristics  and  qualitative  aspects.  Where  an  EC 
is  generating  a  solution,  then  the  solution  and  the  problem 
should  be  presented  together  in  an  honest  and  convincing 
way.  A  major  systems  design  factor  for  the  crew  interface  is 
the  presentation  of  integrated  information,  ‘image  making’, 
so  that  information  is  presented  in  such  a  manner  that  the 
solution  is,  as  far  as  possible,  self-evident. 
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1.  Summary 

The  ground  based  Tactical  Simulator, 
working  in  a  Dasa  Enhanced  DIS,  standard 
DIS,  or  „Stand  alone“  with  individual  train¬ 
ing  tasks.  In  all  configurations  are  a  high 
number  of  intelligent,  unmanned,  full  mis¬ 
sion  virtual  players  responsible  for  a  high 
fidelity  Combat  Szenario. 

These  unmanned,  full  mission  virtual  play¬ 
ers  will  also  be  able  to  communicate.  The 
number  of  participants  include  all  force 
packages  which  are  required  for  realistic 
training. 

Scenario  Editing,  real-time  Scenario  Man¬ 
agement  and  Monitoring,  presented  on  a 
Tactical  Display  (god  eyes  view)  or  a 
3D  view  attachable  to  each  entity,  as  well  as 
recording  and  replay  is  available  for  Exer¬ 
cise  Management. 

Simulation  of  Fighter  Controller  Working 
Position  is  also  included. 

The  tactical  training  system  for  air  crews 
was  presented  for  the  first  time  at  the  ITEC 
exibition,  Lausanne,  1997,  showing  an  ISDN 
connection  of  2  simulator  cockpits,  one  at 
Lausanne,  one  at  Ottobrunn. 

2.  Introduction 

Due  to  the  fact  that  weapon  systems  get 
more  and  more  performance,  the  pilots  can 
no  longer  fullfill  their  training  with  only  real 
flying.  The  flight  hour  of  the  weapon  system 
is  very  expensive,  the  use  of  weapons  only 
allowed  in  a  few  special  areas.  Training  to¬ 
gether  with  other  pilots  in  complex  scenarios 
(ComAO)  happens  not  very  often,  for  some 
pilots  never. 


On  the  other  hand  complexity  and  perform¬ 
ance  of  the  weapon  systems  is  raising  rap¬ 
idly. 

To  solve  this  problem  the  tactical  training 
system  for  air  crews  is  in  development  by 
Dasa. 

3.  Meaning  of  Tactical  Training  System 

The  tactical  training  system  for  air  crews  is  a 
ground  based  training  simulator  with  a  net¬ 
work,  connecting  different  crews  (airborne 
weapon  systems,  AW  ACS,  fighter  controller 
station,  including  air  to  air  weapons,  air  to 
ground  weapons)  into  complex  training  sce¬ 
narios. 

Only  those,  who  have  to  be  trained  are  real, 
all  others  are  simulated  (virtual  players). 

The  system  can  be  used  for  education,  re¬ 
fresher  training,  mission  rehearsal,  learning 
or  developing  new  tactics,  and  weapon  sys¬ 
tem  ratings. 

It  is  also  possible  to  have  autonomous  single 
pilot  cockpits,  which  are  powerful  enough  to 
run  complex  scenarios  due  to  the  virtual 
players. 

4.  Advantages  of  simulator  training 

Only  trainees  who  have  to  be  trained  are 
„flying“,  other  entities  are  simulated.  Ex¬ 
treme  manoeuvres  can  be  trained  (Head  on 
attack/passes  (<20°  ),  close  combat),  if  the 
visual  system  is  appropriately  selected..  It  is 
possible  to  train  new  tactics  with  new  weap¬ 
ons  like  in  reality,  but  without  risk. 

Multi  Bogey  Environments  (whenever  re¬ 
quired,  not  only  once  a  year  or  less),  which 
means  a  higher  number  of  entities  than  in 
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real  flight  training,  totally  independent  from 
season,  weather  and  location.  Besides  the 
advantage  that  training  effectiveness  could 
be  easily  evaluated,  training  costs  are  drasti¬ 
cally  reduced. 

Network  to  other  existing  simulators  and/or 
in  future  a  closed  loop  with  real  flights 
(weapon  system  simulation  in  flight)  is  an 
option  of  this  system. 

5.  System  description 
5.!  Network 

The  system  is  connected  via  standard  ISDN 
telephone  lines  world  wide.  Typically  3-4 
ISDN  lines  (64kBit/s  each)  are  required 
from  cockpit  to  cockpit  for  a  scenario  of  30- 
40  entities. 

The  protocol  used  is  the  Dasa  enhanced  DIS 
protocol.  The  system  can  optionally  be 
driven  with  DIS  or  in  future  under  HLA. 
Dasa  also  has  experience  with  the  German 
and  US  connection  standards. 

The  performance  of  the  network  allows  dy¬ 
namic  data  exchange  in  real  time,  without 
visible  stepping. 

5.2  Virtual  Players 

A  virtual  player  is  a  synthetic  combination 
of  pilot  and  his  aircr^t/weapon  system,  with 
individual,  autonomuos,  intelligent  behavior. 
Each  virtual  player  has  its  individual  simu¬ 
lation  task,  he  decides  on  his  attacks  based 
on  the  knowledge  of  the  weapon  system 
strategy).  Virtual  Players  can  be  friends  or 
enemies,  from  hannless  to  aggressive. 
Speech  recognition  and  synthesis,  for  com¬ 
munication  between  synthetic  and  real  pilot 
allows  the  real  pilot  to  give  commands  to  the 
simulated  wing  man,  so  that  he  cooperates 
with  the  real  pilot. 

Different  weapon  systems  are  available 
(Euro-fighter,  Tornado,  MIG  29),  others  are 
available  on  request  (Grippen,  Mirage, 
F4,....) 


5.3  Fighter  Controller  Station 
This  station  is  the  simulation  of  the  working 
position  of  the  fighter  controller.  All  neces¬ 
sary  data  links  between  this  station  and  the 
simulated  and  real  entities  provide  the  tacti¬ 
cal  situation  to  the  controller. 


5.4  Simulator  Control  and  Mission 
Preparation  (Scenario  Manager  Station) 


The  Real-time  Scenario  Management  and 
Monitoring  is  presented  on  a  tactical  display 
(god  eyes  view),  alternatively  a  3D  view  can 
be  attached  to  each  entity. 

Tools  for  scenario  editing  and  mission  plan¬ 
ning  are  available,  as  will  be  editors  for  en¬ 
vironment  such  as  weather  conditions  (fog, 
rain,  hail,  snow,  thunderstorm). 

The  system  also  allows  simulator  control 
with  the  functions,  start,  stop,  freeze,  con¬ 
tinue,  record,  re-play,  malfunction  simula¬ 
tion  and  emergency  initialisation 
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6.  Future  enhancements 

The  system  will  also  get  links  to  real  air¬ 
crafts  and  those  which  are  using  „weapon 
system  simulation  in  flight"  a  future  product 
of  Dasa. 
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SUMMARY 

This  project  concerns  the  development  of  a 
collaborative  human-electronic  capability  for 
management  of  aircraft  mechanical  faults. 
Health  and  usage  monitoring  systems 
(HUMS)  have  been  developed  for  application 
in  rotorcraft  operations  to  address 
identification  of  fault  conditions  and  tracking 
of  system  usage  relative  to  component  life 
expectations.  HUMS  systems  employ 
various  sensors  (e.g.,  oil  pressure  and 
temperature,  torque,  vibration)  for  engines, 
drive  train,  and  rotors,  along  with  processors 
to  identify  critical  conditions.  The  present 
work  addresses  the  development  of  alerting 
information  for  the  aircrew  regarding 
HUMS-identified  mechanical  faults. 
Focusing  on  the  Navy’s  aging  fleet  of  H-46 
aircraft,  we  have  conducted  an  experimental 
determination  of  aircrew  information  needs 
for  fault  management.  We  are  developing  an 
intelligent  interpretive  layer  on  top  of 
separately  provided  diagnostic  and 
prognostic  algorithms  to  aid  the  aircrew  in 
accomplishing  fault  management  functions 
such  as  corroboration,  impact  assessment, 
and  action  determination.  A  draft  aiding 
concept  and  aircrew  interface  will  be 
presented.  We  will  also  address  the  critical 
questions  of  "When  is  indicator  reliability 
good  enough  to  present  to  the  aircrew?"  and 
"How  can  we  best  design  an  interface  and 
decision  aid  to  help  the  aircrew  to  deal 
effectively  with  inherently  uncertain  warning 
information?" 

INTRODUCTION 


Health  and  usage  monitoring  systems 
(HUMS)  involve  the  application  of  a  variety 
of  advanced  industrial  equipment  monitoring 
technologies  to  helicopter  mechanical 
systems.  HUMS  systems  have  been 
developed  and  implemented  by  several 
vendors  over  the  past  decade  in  order  to 
provide  improved  diagnostic  data  for  ground- 
based  maintenance  operations  (summary 
reviews  of  HUMS  technology  are  available 
in  Stevens,  Hall,  &  Smith,  1996;  Parry,  1996, 
Marsh,  1996).  The  majority  of  HUMS 
installations  to  date  have  been  made  by  UK 
and  Norwegian  operators  who  provide 
helicopter  ferrying  services  to  North  Sea  oil 
platforms.  These  installations  have  been 
made  in  order  to  improve  flight  safety,  and 
have  been  quite  successful  in  this  regard. 

The  U.S.  Navy  has  recently  conducted  an  Air 
Vehicle  Diagnostic  Systems  (A YDS) 
program  (Chamberlain,  1994)  in  order  to 
investigate  the  potential  benefits  of  HUMS 
technology  to  improve  safety  and 
maintenance  in  aging  rotorcraft  fleets 
(particularly  H-53,  H-46,  and  SH-60)  and  to 
facilitate  the  Navy's  transition  to  a  paradigm 
of  "condition-based  maintenance"  (CBM). 
CBM  is  intended  to  replace  the  current 
usage-based  maintenance  policy  whereby 
major  maintenance  actions  and  component 
replacements  are  scheduled  according  to 
recorded  aircraft  component  usage  (typically 
flight  hours,  operating  hours,  takeoffs, 
landings,  etc.)  as  compared  to  expected 
component  usage  life.  Since  usage-based 
maintenance  policies  are  necessarily  quite 
conservative,  a  conversion  to  a  CBM  policy 
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is  expected  to  produce  significant  savings  in 
maintenance  costs  along  with  an 
improvement  in  aircraft  safety. 

Current  HUMS  systems  do  an  excellent  job 
of  monitoring  and  recording  usage  data  in 
fine  detail  so  as  to  enable  improvements  in 
usage-based  maintenance.  However,  in  the 
area  of  health  monitoring,  current  HUMS 
capabilities  are  more  limited.  While  many 
mechanical  faults  can  be  reliably  identified, 
many  others  are  quite  difficult  to  diagnose. 
For  health  monitoring,  interest  has  focused 
particularly  on  the  use  of  vibration  sensors 
(i.e.,  accelerometers)  which  are  located 
around  key  drive-train  components. 

Vibration  data  are  especially  important  for 
monitoring  gearboxes  for  which  (unlike 
engines)  there  are  very  little  other  indicator 
data.  A  variety  of  techniques  have  been 
developed  to  analyze  and  interpret  HUMS 
vibration  data.  One  approach  employs  higher 
order  (e.g.,  fourth  or  sixth  moment)  feature 
vectors  that  are  derived  from  the  raw 
vibration  data,  and  then  compared  with 
norms  for  those  feature  vectors  in  order  to 
determine  when  an  anomaly  exists.  Another 
recently  emerging  approach  uses  neural 
network  algorithms  to  compare  raw  or 
processed  vibration  data  with  patterns  for 
known  faults,  and  so  achieve  diagnoses. 

Both  of  these  approaches  require 
considerable  analytical  and  subjective 
judgment  on  the  part  of  the  HUMS  expert. 

In  the  particularly  critical  and  challenging 
area  of  rotorcraft  gearboxes,  high  false  alarm 
rates  (in  the  vicinity  of  20%  to  40%)  are 
reported  for  returns  of  non-faulty  gearboxes 
to  the  manufacturer. 

HUMS  technology  appears  to  have 
considerable  potential  for  alerting  the  aircrew 
of  in-flight  developments  of  mechanical 
faults.  This  situation,  however,  presents  a 
rather  unique  and  challenging  opportunity  in 
human-centered  design.  While  the 
underlying  technology  is  evolving  rapidly 
and  high  levels  of  uncertainty  are  associated 


with  many  sensor  patterns  and  fault 
conditions,  there  are  also  immediate 
opportunities  to  provide  valuable  lead-time 
warnings  to  aircrews  of  impending 
catastrophic  mechanical  failures.  While  it 
may  be  tempting  to  wait  for  the  HUMS 
technology  to  mature  before  designing  an 
aircrew  interface,  the  potential  near-term 
safety  benefit  is  more  compelling.  Of  course, 
a  thorough  cost-benefit  analysis  must  be 
conducted  and  an  appropriate  aircrew 
interface  design  must  be  developed  before 
any  HUMS  data  are  presented  to  the  aircrew. 
The  remainder  of  this  paper  is  concerned 
with  the  aircrew  interface  design  process. 

INTERFACE  DESIGN 

The  aircrew  interface  design  problem 
consists  of  determining  what  information  to 
present  and  how  to  present  it.  We  will  focus 
mainly  on  the  former  of  these  issues  because 
it  is  quite  difficult  and  still  not  fully  resolved, 
and  it  is  logically  prerequisite  to  the  latter 
information  presentation  issue  for  which 
fairly  well-established  principles  and 
techniques  are  available. 

In  order  to  determine  what  information 
should  be  presented  to  the  aircrew,  it  is 
necessary  to  establish  both  what  information 
the  aircrew  needs  and  wants  and  also  what 
information  can  feasibly  be  generated  to 
satisfy  their  needs/desires.  Since  essentially 
none  of  the  currently  generated  HUMS 
outputs  are  expected  to  be  suitable  for 
aircrew  presentation,  this  process  is  expected 
to  require  iterative  refinement.  We  start  by 
identifying  the  general  types  of  information 
that  the  aircrew  needs,  then  we  try  to 
characterize  the  kinds  of  information  that  we 
think  we  can  generate  in  the  identified 
categories,  then  we  assess  the  utility  to  the 
aircrew  of  those  specific  kinds  of 
information,  etc.  We  assume  that  all 
information  that  might  be  presented  to  the 
aircrew  would  have  to  be  generated  via  some 
kinds  of  aiding  algorithms  which  would  use 


the  HUMS  data  as  their  primary  inputs.  Such 
algorithms  could  vary  from  extremely  simple 
ones  like  current  chip  detector  lights  which 
just  indicate  that  some  anomaly  has  been 
detected,  to  very  sophisticated  algorithms 
which  would  tell  the  aircrew  precisely  what 
to  do  to  respond  optimally  to  the  detected 
problem. 

INFORMATION  REQUIREMENTS 

Both  analytical  and  empirical  approaches  to 
identification  of  aircrew  information  needs 
are  warranted.  The  analytical  approach  is 
needed  because  it  is  natural  to  expect 
aircrews  to  be  somewhat  fixated  on  types  of 
information  with  which  they  are  already 
familiar  in  current  cockpit  interfaces,  so  they 
may  not  think  to  ask  for  some  new  types  of 
information  that  might  be  very  helpful.  So 
we  should  attempt  analytically  to  identify  all 
potentially  relevant  types  of  information. 

But  it  is  also  important  to  determine  what 
information  the  aircrew  would  actually  use  if 
it  were  available  because  it  could  be 
detrimental  to  present  information  that  they 
won't  use,  and  it  is  likely  to  be  quite  costly  to 
create  many  of  the  algorithms  for  generating 
the  candidate  aiding  information. 

Our  effort  to  develop  a  taxonomy  of  relevant 
aircrew  information  needs  began  with  an 
earlier  investigation  of  the  rotorcraft 
waming-caution-advisory  (WCA)  design 
problem  (Hicinbothom  et  al.,  1995).  We 
extended  the  earlier  taxonomy  to  reflect 
HUMS  considerations  to  produce  the 
taxonomy  illustrated  in  Table  1.  This 
taxonomy  includes  two  types  of  information 
-  primary  and  qualifying  information.  We 
assume  that  the  aiding  information  is 
constructed  on  top  of  a  layer  of  HUMS  data 
and  information.  The  aiding  information 
serves  to  convert  the  HUMS  data  outputs  into 
information  that  addresses  the  various 
aircrew  information  needs.  We  have 
envisioned  a  sort  of  hierarchy  of  levels  of 
information  progressing  from  anomaly 


identification  through  diagnosis,  prognosis, 
impact  assessment,  and  leading  ultimately  to 
action  recommendation.  Also  in  the  aiding 
information  layer,  but  orthogonal  to  the 
preceding  hierarchy  are  several  types  of 
qualifying  information  such  as  corroboration, 
criticality,  urgency,  and  uncertainty,  which 
can  apply  to  all  levels  in  the  hierarchy. 


Primary  Information 

Anomaly  detection  and  localization 

Diagnosis 

Prognosis 

Root  cause  determination 
Impact  assessment 
Action  determination 

Qualifying  Information 

Corroboration 

Criticality 

Urgency 

Uncertainty 

Precision 

Completeness _ 

Table  1 .  Mechanical  Fault  Alerting 
Information  Taxonomy 

Current  HUMS  systems  provide  some  data 
on  anomaly  identification  and  diagnosis,  but 
none  of  this  is  currently  considered  suitable 
to  pass  directly  on  to  the  aircrew.  There  are 
ongoing  research  programs  to  develop 
diagnostic  and  prognostic  information  from 
vibration  data,  but  the  quality  and  timing  of 
results  in  these  areas  are  questionable. 
Various  projections  have  been  made  of 
expected  developments  over  the  next  five 
years  for  the  precision  and  reliability  of 
HUMS  diagnostic  and  prognostic 
capabilities.  While  HUMS  developers  may 
provide  some  data  suitable  for  cockpit 
presentation  in  the  next  5  to  10  years,  we 
must  assume  that  most  of  the  information  to 
be  offered  to  the  aircrew  will  have  to  be 
developed  through  some  kind  of  processing 
that  will  operate  on  top  of  HUMS.  We 
envision  production  rule  (or  expert)  systems. 
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possibly  with  some  model-based 
components,  to  be  the  primary  mechanisms 
for  information  generation  in  most  of  the 
identified  categories.  Although  it  is  certainly 
desirable  that  the  production  rules  and 
models  be  based  as  much  as  possible  on  solid 
analytical  and  empirical  data,  it  is  expected 
that  the  initial  formulation  of  many  of  the 
rules  will  have  to  be  derived  by  knowledge 
elicitation  from  subject  matter  experts  (i.e., 
by  subjective  estimation  and  speculation).  In 
the  area  of  action  determination,  we  expect 
that  this  approach  is  ultimately  the  only 
option,  since  a  value  decision  must  be  made 
about  the  tradeoffs  between  the  various 
mission  outcome  possibilities.  For  action 
recommendations,  we  are  essentially 
producing  a  sort  of  electronic  NATOPS 
which  must  naturally  incorporate  all  existing 
NATOPS  and  be  fully  compatible  with  it. 

But  NATOPS  currently  only  addresses 
conditions  and  anomalies  that  can  be 
identified  with  current  sensors.  HUMS 
greatly  expands  the  universe  of  cases  that 
must  be  addressed. 

It  is  also  important  to  note  that  not  all 
information  needs  are  necessarily  event 
triggered.  Some  information  requests  and 
system  interactions  may  occur  at  the 
initiative  of  the  aircrew  for  various  reasons 
such  as  the  following: 

•  command  of  recording  events  - 
Current  HUMS  systems  are  memory 
limited  and  cannot  record  data  from  all 
sensors  continuously,  especially  from 
vibration  sensors  which  produce 
particularly  large  data  volumes. 
Intermittent  recordings  are  currently 
triggered  by  key  flight  events  (e.g., 
take-off  and  landing)  as  well  as  by 
cyclic  schedules,  but  these  systems 
also  allow  the  aircrew  to  trigger 
recordings  based  on  aircrew  concerns 
about  system  performance. 

•  command  to  perform  special  analyses 
-  As  sophisticated  options  for 
processing  and  viewing  data  continue 


to  develop,  we  expect  the 
corresponding  decisions  to  be  afforded 
to  the  aircrew. 

•  review  of  subsystem  status 
data/indicators  —  We  also  expect  to 
provide  the  aircrew  with  options  to 
examine  both  current  and  historical 
records  of  all  subsystem  status 
indicators. 

•  trouble  reporting  —  Trouble  reports 
could  be  initiated  by  the  aircrew  and 
recorded  so  as  to  facilitate  ground 
crew  analysis  in  conjunction  with  all 
corresponding  HUMS  data. 

In  order  to  develop  an  empirical  perspective 
on  aircrew  information  requirements,  a  study 
was  conducted  with  H-46  aircrews  at  NAS 
North  Island  using  a  motion-base  simulator 
in  order  to  establish  baseline  performance  of 
aircrews  confronted  with  a  variety  of 
mechanical  faults  and  current  cockpit 
instrumentation.  Three  ten-minute  flight 
segments  were  conducted  by  eight  variously 
experienced  aircrews  (pilot,  copilot,  and  crew 
chief)  in  the  context  of  an  embassy  personnel 
extraction  scenario  with  one  opportunistic 
search-and-rescue  operation.  Performance 
data  were  collected  via  videotape, 
questionnaire,  and  extensive  videotaped 
debriefs.  Objective  and  subjective  data  were 
analyzed  to  identify  areas  of  attentional 
focus,  types  of  communications,  any  evident 
problems,  and  types  of  desired  aiding.  The 
results  of  this  study  generally  supported  the 
above  described  taxonomy  of  information 
needs.  Participants  reported  dividing  their 
attention  roughly  equally  across  the 
categories  of  display  accuracy,  diagnosis, 
prognosis,  impact  assessment,  and  action 
recommendation.  However,  in  classifying 
recorded  voice  conununications  during 
periods  when  the  malfunction  indications 
were  present,  we  observed  many  more 
comments  in  the  category  of  action 
recommendation  (28%)  than  in  diagnosis 
(3%)  or  prognosis  (4%),  while  a  substantial 
number  (7%)  were  concerned  with 
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corroboration  of  indications.  Thus,  it  is  clear 
that  aircrews  are  concerned  with  all  of  the 
identified  categories  of  information  at  one 
time  or  another.  More  complete  reports  of 
this  study  are  in  preparation  for  presentation 
at  an  upcoming  conference  (Deaton  et  al., 
1997  a  &  b). 

There  is  also  a  separate  issue  of  information 
timing  that  must  be  addressed  in  interface 
design.  The  HUMS  processor  will  typically 
develop  a  fault  analysis  over  time  and  initial 
results  may  be  modified  or  elaborated  as 
additional  data  is  obtained.  It  may  be 
possible  and  appropriate  to  notify  the  aircrew 
of  a  significant  anomaly  prior  to  the 
development  of  additional  diagnostic  and 
prognostic  information.  Or,  in  other  cases,  it 
may  be  more  appropriate  to  withhold 
presentation  of  any  information  until  a 
complete  profile,  including  action 
recommendations,  ean  be  provided.  And 
after  a  partial  or  full  profile  of  information  is 
viewed  by  the  aircrew,  it  may  be  important  to 
notify  them  of  subsequent  changes  to  aspects 
of  that  profile.  Clearly,  we  must  develop 
some  means  of  determining  when  partial  or 
modified  information  will  be  sufficiently 
relevant  and  urgent  to  the  aircrew  to  offset  its 
potentially  distracting  effects. 


Figure  1  illustrates  the  general  concept  that 
we  have  for  aiding  the  aircrew.  A  special 
processor  will  perform  information  analysis 
for  the  aircrew,  taking  inputs  from  available 
HUMS  data  as  well  as  other  avionics  and 
mission  plan  data.  Information  will  be 
provided  to  the  aircrew,  via  some  new 
Warning-Caution- Advisory  (WCA)  system 
interface,  in  categories  such  as  condition 
assessment,  corroboration,  and  action 
recommendations.  Our  preliminary  design 
calls  for  initial  indications  of  anomalies  to  be 
signaled  by  a  dedicated  WCA  light  (e.g., 
labeled  “DRIVE  TRAIN”)  which  then  allows 
the  aircrew  to  request  elaboration.  The 
request  for  elaboration  leads  to  a  display  of 
summary  information  about  the  anomaly 
(e.g.,  symptoms,  locus,  severity,  etc.)  which 
also  includes  options  to  branch  to  further 
elaborations  in  the  areas  of  diagnosis, 
prognosis,  action  recommendations,  and 
corroboration.  We  also  expect  that  it  may  be 
appropriate  to  offer  somewhat  different 
information  to  different  crewstations,  such  as 
to  support  the  distinct  roles  of  the  pilot  and 
crew  chief  in  the  H-46.  A  prototype  aiding 
interface  is  currently  being  designed  and 
implemented  for  evaluation  in  a  second  study 
in  the  H-46  simulator  at  NAS  North  Island. 


Figure  1.  Aircrew  Aiding 
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CONCLUSIONS 

Continuing  efforts  are  investigating  the  types 
of  and  constraints  on  information  that  can 
feasibly  be  generated  and  presented  in  each  of 
these  categories.  Decision  and  information 
needs  are  being  identified  for  each  of  the  crew 
positions  in  order  to  develop  interface  designs 
that  are  tailored  to  the  needs  of  each 
crewmember.  Issues  of  crew  coordination  and 
training  are  also  being  addressed.  This  work 
is  closely  coordinated  with  ONR-sponsored 
work  on  diagnostics  and  prognostics  of 
mechanical  systems  and  associated  condition- 
based  maintenance  methods. 

At  the  same  time,  there  are  some  substantial 
challenges.  The  kind  of  intelligent  WCA  aid 
that  we  would  like  to  develop  can  be  most 
easily  developed  in  a  new  aircraft  with 
advanced  computers,  reconfigurable  displays, 
and  a  digital  data  bus.  But  the  need  is  greatest 
in  the  aging  rotorcraft  fleets,  like  the  H-46 
which  doesn’t  offer  the  desired  infrastructure. 
Also,  the  underlying  HUMS  technology  and 
related  technologies  for  diagnosis  and 
prognosis  are  developing  rapidly.  There  is  a 
temptation  to  wait  for  these  technologies  to 
mature,  but  there  is  also  the  concern  that  the 
need  for  aiding  is  now.  Thus,  for  example,  we 
must  weigh  the  high  uncertainties  we  now 
have  in  vibration-based  prediction  of  gearbox 
failures  against  the  catastrophic  results  of  in¬ 
flight  failures.  We  must  also  recognize  that 
information  needs  probably  vary  with  many 
other  factors  which  we  have  yet  to  investigate. 
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SUMMARY 

Research  on  and  development  of  an  electronic 
crewmember  (EC)  has  matured  to  allow  a 
transition  from  the  scientific  world  to  practical 
applications  in  the  near  future.  With  this 
transition,  the  implementation  in  a  real  world 
environment  reveals  new  problems  beyond  the 
successful  application  of  artificial  intelligence 
technology.  At  the  end  of  the  day,  the  electronic 
crewmember  has  to  be  certified. 

This  paper  considers  certification  issues  of  the 
EC.  Starting  point  is  the  definition  of  two  generic 
EC  models  and  their  interaction  with  the  on-board 
environment.  Within  this  text,  the  first  of  these 
models  is  called  the  EC  as  adviser.  The  second 
one  comprises  the  role  of  a  supervisory  controller. 
Although  a  real  EC  implementation  will  probably 
not  match  one  of  these  roles  exactly,  this 
distinction  served  well  during  the  generation  of 
this  paper  and  may  also  assist  the  reader  to 
become  sensitive  for  the  criticality  of  electronic 
crewmember  functions. 

Next,  safety  concerns  regarding  the  EC  are 
discussed.  From  a  common  understanding  of  the 
applicable  criticality  criteria,  safety  relevant 

features  of  both  EC  models  are  derived.  The 
analysis  shows  that  the  safety  impact  of  the  EC  is 
mainly  related  to  human  factors,  especially  the 
perception  of  the  electronic  crewmember  by  the 
human  colleagues. 

Detailed  techniques  to  verify  and  validate  the 
implementation  of  expert  systems  and  neural 
networks  are  not  discussed  due  to  the  limited 
experience  of  the  author  with  artificial 

intelligence.  But  what  is  shown  is  a  route  to 
certification  based  on  the  qualification  and 
certification  experience  with  safety  critical 

systems,  in  particular  flight  control  systems 

(FCS). 


safety  concerns  from  above  are  compared  with 
equivalent  FCS  design  characteristics.  Finally, 
verification  and  validation  methods  that  may  be 
suitable  in  the  EC  certification  process  arc  derived 
from  the  equivalent  methods  used  in  FCS 
certification. 


1.  INTRODUCTION 

Automation  of  aircraft  functions  has  a  long 
history.  Around  1930  research  on  autopilots  had 
achieved  a  maturity  that  allowed  their 
implementation  in  production  aircraft  [2].  This 
was  the  first  shift  to  change  the  pilot’s  role  from 
the  active  flight  path  controller  to  a  supervisory 
role.  The  reasons  for  the  introduction  of  autopilots 
might  have  been  similar  to  those  from  which 
modern  flight  decks  have  emerged: 

•  automation  of  functions  that  an  automatic 
controller  can  perform  more  precisely 

•  introduction  of  functions  that  a  pilot  is  unable 
to  perform 

•  pilot’s  workload  reduction  that  enables  him  to 
concentrate  on  other  tasks 

As  a  result  a  more  reliable  and  safer  service  was 
promised  for  commercial  transport  airplanes  while 
higher  mission  accomplishment  rates  were  an 
additional  driving  factor  for  military  aircraft. 
However,  automation  generates  safety  threats  by 
itself: 

•  Implementation  faults  have  led  to  accidents. 

•  The  equipment  by  which  the  automation 
functions  were  implemented  failed  and  created 
hazards. 

•  Specified  functionality  occasionally  did  not 
meet  the  original  intentions  and  created 
catastrophic  aircraft  states. 

Thoroughly  performed  design,  implementation  and 
verification  of  equipment  functions  are  the  means 
for  protection  against  implementation  faults. 
Sound  procedures,  checklists  and  independent 


To  get  an  initial  understanding  how  an  EC  may  be 
certified  a  comparation  technique  is  chosen.  EC 
design  features  imposed  by  the  EC  models  and  the 
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verification  are  the  proven  methods  commonly 
used. 

To  mitigate  possible  effects  of  equipment  failure, 
warning  annunciations  to  the  pilot  provide  the 
necessary  situational  awareness.  A  direct  relation 
between  annunciation  and  required  pilot’s  action 
allowed  hazard  recovery  even  when  immediate 
action  is  required.  To  prolong  the  reaction  time 
for  successful  recovery,  measures  like  trim 
actuators  with  limited  maximum  travel  speed  are 
commonly  used.  Or,  if  this  is  not  feasible  as  it  is 
for  stability  augmentation  systems,  the  authority 
of  the  automatic  mode  is  only  a  small  fraction  of 
the  full  travel  range  to  limit  the  maximum  failure 
transient. 

In  cases  where  the  pilot  is  not  able  to  counteract 
the  failure  transient,  fail-safe  devices  were 
invented.  Fail-safe  techniques  have  been  the  first 
step  to  implement  similar  and  dissimilar 
redundancy.  The  redundant  means  are  used  to 
assess  the  outputs  of  the  automatic  controller  for 
validity.  Thus,  maximum  failure  transients  are 
limited  automatically  to  an  extent  that  the  pilot 
can  perform  safe  recovery  without  exceptional 
skills. 

For  essential  equipment  for  which  uninterrupted 
performance  is  required  like  digital  flight  control 
systems,  fail-safe  characteristics  are  not  sufficient. 
Instead,  fail-operativeness  to  the  probability  level 
specified  for  safety  is  mandatory.  Sophisticated 
redundancy  mangement  techniques  are  the  basis  to 
fulfil  fail-operative  requirements. 

Today,  aircraft  systems  have  the  tendency  to  be 
rather  complex,  especially  when  digital  computers 
comprise  the  major  portion  of  the  functionality. 
This  challenges  design  and  verification  skills.  Up 
to  now,  no  sound  techniques  are  available  to  prove 
all  presumptions  and  intentions  sufficiently  in 
addition  to  the  verification  of  the  requirements 
written  in  a  system  specification.  This  is  a  source 
of  error  that  can  reduce  system  safety  dramatically 
due  to  malfunctions  of  single  systems  and  even 
more  when  the  interaction  of  several  systems  is 
considered. 

The  pilot  has  to  cope  with  these  complex  systems 
in  normal  operating  modes  as  well  as  with  remote 
hazardous  failure  modes.  The  type  of  system 
management  required  has  shifted  from  fixed  and 
straight  recovery  procedures  to  recovery  processes 
that  need  not  only  immediate  action,  but  also  an 
assessment  of  complex  system  states  and 
prioritization  of  tasks. 

Because  humans  are  less  suited  for  the  latter, 
electronic  crew  members  are  promising  means  for 


pilots’  assistance.  However,  the  electronic 
crewmember  by  itself  is  a  technical  system  and  is 
prone  to  errors  during  all  life  cycle  phases  like 
other  systems  of  similar  complexity. 


2.  SYSTEM  ARCHITECTURE 

2.1  The  Electronic  Crew  Member  as  Adviser 

Figure  1  shows  a  functional  interface  diagram  of 
the  pilot,  the  off-board  environment,  the  aircraft, 
the  aircraft  systems  and  the  EC  in  an  advisory 
role. 


Fig.  1:  The  Electronic  Crewmember  as  Adviser. 

The  pilot  interacts  with  the  off-aircraft 
environment  by  visual  cues.  Furthermore,  he  is 
communicating  with  air  traffic  control  or  tactical 
control  centers.  With  respect  to  the  aircraft 
performance  and  dynamics  he  also  is  sensing,  e.  g. 
using  his  inertial  and  tactile  senses  as  well  as  the 
flight  displays.  Regarding  the  aircraft  systems,  he 
has  a  monitoring  role  as  well.  Let  us  define 
monitoring  as  an  integrity  assessment  of  the 
aircraft  systems.  The  information  provided  by 
these  means  enables  the  pilot  to  perform  his  role 
as  a  supervisory  controller  of  the  aircraft 
performance  and  dynamics  states.  He  also  controls 
the  aircraft  systems  by  mode  selection. 

The  electronic  crewmember  in  an  advisory  role 
does  not  challenge  the  pilot  as  supervisory 
controller.  Instead  the  electronic  crewmember  may 
have  the  ability  to  alleviate  concerns  in  two  major 
areas  regarding  cockpit  automation  as  stated  in 
[1]: 

•  Pilot  understanding  of  the  automation,  its 
capabilities,  behavior,  modes  of  operation,  and 
procedures  for  use,  and 

•  Differing  pilot  decisions  about  the  appropriate 
automation  level  to  use  (if  any)  in  normal  and 
non-normal  circumstances. 
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For  the  assessment  of  the  actual  situation  the  EC 
gathers  information  from  the  aircraft  systems  and 
the  pilot’s  behaviour.  The  correctness  of  the 
information  is  not  questioned  by  the  EC.  From  his 
knowledge  base  that  is  naturally  different  from  the 
pilot’s  knowledge  base  conclusions  are  derived 
regarding  status  accounting  and  priorisation  of 
tasks.  The  pilot  is  free  to  follow  the  EC’s 
proposals  or  to  reject  it. 

With  respect  to  integrity  and  safety  the  advantages 
of  the  electronic  crewmember  as  adviser  lie  in  the 
following  areas: 

•  The  EC  does  not  interfere  with  the  redundancy 
management  of  the  aircraft  systems.  This 
makes  design  and  qualification  easier  because 
failure  modes  of  the  aircraft  systems  that  go 
beyond  defined  normal  and  abnormal  operating 
modes  have  not  to  be  considered. 

•  The  pilot  cannot  always  be  sure  that  the  EC  is 
right  with  his  proposals.  That  may  limit  the 
reliance  on  this  advice.  Hence  the  safety 
impact  may  be  limited.  From  flight  tests  with 
CASSY  [4]  some  indications  are  available  that 
pilots  may  be  able  to  cope  with  such  kind  of 
crew  assistant. 

However,  the  mentioned  advantages  may  ease 
qualification  and  certification,  but  there  are  some 
safety  risks  associated  with  the  advisory  role: 

•  Regarding  the  EC’s  interface  with  the  aircraft 
systems,  masking  of  hazardous  states  by  the 
aircraft  systems  may  become  critical.  The 
masking  problem  is  typical  for  automatic 
control  loops.  When  a  failure  occurs  the 
control  loop  compensates  for  the  failure 
transients  up  to  its  authority  limits.  When  the 
authority  limit  is  reached  no  further 
compensation  is  possible,  the  aircraft  state  is 
far  from  equilibrium,  and  large  failure 
transients  may  lead  to  hazardous  situations  [1]. 
To  what  extent  the  EC  as  adviser  is 
contributing  or  mitigating  failure  masking  has 
to  be  analyzed  for  each  particular  EC  function. 

•  In  [4]  it  is  reported  that  discrepancies  between 
pilots  and  CASSY  are  caused  by  pilot  error  or 
by  machine  error.  Both  figures  are  in  the 
percent  range.  As  long  as  pilot  machine  errors 
are  not  remote  pilot’s  reliance  on  the  EC  may 
be  limited.  Because  this  assessment  process 
takes  time,  an  EC  of  this  kind  is  unable  to 
assist  the  pilot  in  cases  when  immediate  action 
is  required.  However,  it  can  be  expected  that 
the  EC  will  be  improved.  Thus  machine  error 
probability  will  be  order  of  magnitudes  smaller 
than  pilot  error  probability.  A  role  change  to 
the  EC  as  supervisory  controller  may  be 
imposed. 


2.2  The  Electronic  Crew  Member  as 
Supervisory  Controller 

Figure  2  shows  the  EC  as  supervisory  controller. 
The  figure  is  only  slightly  different  from  figure  1. 
Therefore,  only  the  differences  and  their 
consequences  are  discussed. 


Fig.  2:  The  Electronic  Crewmember  as 
Supervisory  Controller. 

In  this  role  the  EC  does  not  only  gather 
information  from  the  pilot  and  the  aircraft 
systems,  but  performs  also  a  monitoring  function. 
This  role  requires  some  sort  of  meta  knowledge 
that  enables  the  EC  to  give  control  inputs  to  the 
aircraft  systems  especially  for  mode  control  and  to 
guide  the  pilot  to  perform  particular  actions  in  a 
certain  sequence. 

Theoretically,  the  EC  as  supervisory  controller 
might  be  a  powerfull  function  with  a  high 
potential  for  flight  safety  enhancements.  However, 
it  is  questionable  if  such  a  system  can  be  built  and 
even  more  if  it  can  be  certified. 

The  safety  classification  of  the  monitoring 
function  has  to  be  at  least  the  highest 
classification  of  the  observed  aircraft  systems. 
Because  the  complexity  is  the  major  problem  for 
aircraft  system  design  it  is  not  reasonable  to  cure 
this  problem  with  even  more  complexity. 

In  very  general  terms,  failures  can  be  categorized 
as  expected  and  unexpected.  Expected  failures  are 
those  considered  during  development,  e.  g.  during 
analysis  and  design  or  during  the  verification 
process.  Practical  experience  has  demonstrated 
that  the  probability  of  expected  hardware  failures 
is  less  than  calculated  because  they  are  mitigated 
by  appropriate  means  during  the  design  phase. 
Failure  modes  not  considered  have  the  highest 
potential  to  bring  reliability  down. 
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For  non-hardware  of  any  kind  similar  rules  apply. 
In  [3]  many  well  known  catastrophic  accidents 
with  technical  systems  are  analyzed  demonstrating 
that  unexpected  chains  of  events  are  the  most 
likely  cause  for  hazards  of  complex  technical 
systems.  Another  indication  is  the  fact  that  in 
software  engineering  the  number  of  analysis  and 
design  errors  is  double  as  high  as  the  number  of 
coding  errors  [5].  With  increasing  maturity  of 
coding  standards  it  can  be  expected  that  the  ratio 
gets  even  worse.  Most  of  the  analysis  and  design 
errors  are  usually  detected  during  the  verification 
and  validation  process.  However,  the  most  remote 
errors  may  not  be  identified  during  development 
and  may  have  the  highest  hazard  potential  in 
service. 

One  significant  result  from  these  considerations  is 
that  the  verification  and  validation  problem  of 
complex  systems  is  only  slightly  dependent  of  the 
design  philosophy,  may  conventional  control  laws 
or  may  artificial  intelligence  techniques  be  used. 
However,  a  well  structured  functional  architecture 
and  a  more  linear  behaviour  can  be  better  verified 
than  unstructured  and  extremely  nonlinear 
functions. 

The  functional  interface  between  the  pilot  and  EC 
is  reflecting  the  competition  between  both 
supervisory  controllers.  On  one  hand,  the  concerns 
on  possible  pilot’s  overconfidence  on  the  EC 
apply  as  mentioned  above.  On  the  other  hand,  the 
EC  lacks  the  flexibility  of  the  human  pilot. 
Therefore,  the  role  of  the  supervisory  controller 
should  always  stay  with  the  pilot  and  should  not 
be  shifted  to  the  EC. 

Therefore,  the  EC  as  supervisory  controller  is  not 
considered  any  further  in  this  paper. 

However,  after  all  these  safety  concerns  the 
conclusion  is  not  that  electronic  crewmembers 
should  be  better  not  implemented.  What  should  be 
clear  is  that  safety  respectively  survivability  in 
some  military  applications  is  likely  an  issue  for 
ECs. 


3.  ROUTE  TO  CERTIFICATION 

To  propose  a  route  to  certification  for  an  EC  the 
development  tasks  have  to  be  defined  in  the  first 
step.  In  this  paper  the  engineering  tasks  are 
deduced  from  the  development  steps  that  have  to 
be  performed  for  a  FCS  by  analogy. 

In  the  second  step,  the  commonly  used  verification 
methods  of  the  FCS  development  results  of  each 
engineering  task  are  briefly  described.  By  analogy 


again,  these  verification  methods  are  traced  to  the 
EC  verification  process.  In  some  cases  appropriate 
qualification  methods  can  be  easily  derived  by  this 
comparation  technique.  In  other  areas  further 
research  on  suitable  verification  methods  is 
required.  The  results  presented  in  this  paper  are 
preliminary.  Best  usage  is  to  consider  them  as  a 
starting  point  for  further  discussion. 

For  both  systems  the  engineering  areas  of  human 
factors,  system  functionality  and  equipment  have 
to  be  considered. 


3.1  Human  Factors 

EC  as  well  as  FCS  have  to  follow  human  centered 
design  principles.  Therefore,  the  suitability  for 
service  has  to  be  finally  assessed  by  the  users.  For 
FCS,  pilot  assessment  campaigns  are  repeatedly 
performed  during  the  development  phase.  Starting 
with  handling  qualities  simulation  in  an  early 
stage  pilots  are  involved  in  the  evaluation  of  basic 
functions  and  their  integration  to  the  complete 
FCS.  Cooper-Harper  rating  is  the  assessment 
method  widely  used  for  achieving  comparable 
results.  For  flight  clearance,  the  manoeuvres  flown 
in  the  simulator  exercise  the  actual  manoeuvres  of 
the  particular  test  flight.  During  flight  testing  the 
final  assessments  are  made. 

This  test  method  is  directly  traceable  to  the  EC. 
For  the  EC  handling  qualities  are  related  to  the 
selection,  priorisation  and  presentation  of  the 
EC’s  advice.  However,  a  tailored  rating  scheme 
should  be  used.  It  is  understood  that  various  rating 
methods  are  available.  It  is  important  for  the  EC 
community  to  harmonize  the  different  assessment 
philosophies.  A  common  state  of  the  art  procedure 
should  be  established  that  can  be  used  in  the 
certification  process. 

For  other  ergonomic  FCS  design  features  related 
to  FCS  inceptors,  indications  and  mode  selection 
design  standard  and  common  practices  provide  the 
necessary  guidance.  Compliance  with  standards  is 
therefore  a  prerequisite  for  certification  unless 
good  reasons  exists  for  deviating  from  these 
standards  and  are  accepted  by  the  pilots.  This  is 
quite  often  the  case  for  new  military  aircraft  that 
provide  new  or  enhanced  functionality.  Usually 
hand  on  throttle  and  stick  concepts  (HOTAS)  are 
common  practice  for  military  aircraft. 

Because  the  information  flow  from  the  EC  to  the 
pilot  is  already  covered  by  the  handling  quality 
assessment  the  remaining  ergonomic  issues  are 
related  to  the  information  gathering  activity  of  the 
EC.  For  design  of  this  interface  further  research  is 
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still  required  to  define  appropriate  means  that 
perform  their  function  reliable  in  the  operational 
environment.  It  can  be  expected  that  design 
guidelines  will  be  established  together  with  a 
particular  technology.  It  is  recommended  that 
pilots  are  involved  early  in  this  process. 


3.2  System  Functionality 

The  verification  of  FCS  functionality  covers 
analytic  flying  qualities,  control  law  engineering 
and  graceful  degradation. 

Over  the  years  experience  has  been  gained  that 
good  or  excellent  handling  qualities  are  traceable 
to  certain  characteristics  of  the  non-human  part  of 
the  flight  control  loop.  Hence  analytic  flying 
qualities  criteria  are  the  starting  point  for  control 
law  generation.  With  updated  aircraft  performance 
and  aerodynamic  data  the  control  laws  are  refined 
during  development. 

It  is  pretty  clear  that  the  underlying  models  (e.  g. 
differential  equations)  for  FCS  control  law  design 
are  different  from  the  cognitive  models  of  the  EC. 
Nevertheless,  something  may  be  learned  from  the 
verification  methods  used  for  control  law 
evaluation. 

The  flight  envelope  expansion  of  modern  fighter 
aircraft  makes  it  impossible  to  use  a  single  set  of 
linear  differential  equations  for  modelling  aircraft 
dynamics.  Instead  most  parameters  are  functions 
of  altitude,  mach  number  and  angle  of  attack.  The 
steps  usually  chosen  to  validate  the  control  law 
comprises  the  following  steps: 

•  Stationary  manoeuvre  performance  is  analysed 
first  for  all  relevant  points  of  the  flight 
envelope. 

•  Next,  small  amplitude  disturbances  and  control 
inputs  are  simulated  to  evaluate  the  dynamics 
around  certain  equilibrium  points.  The 
simulation  results  can  be  validated  against 
linear  differential  equations  that  are  valid  for 
the  particular  equilibrium  point. 

•  In  the  last  step  gross  manoeuvring  is  exercised, 
to  monitor  the  performance  of  the  full  non¬ 
linear  system. 

The  features  of  the  EC  equivalent  to  analytic 
flying  qualities  and  control  law  engineering  are 
the  control  strategy  and  the  outcome  of  the 
knowledge  base  engineering  process.  From  an 
outside  view  the  EC  can  still  be  considered  as  a 
hopefully  deterministic  non-linear  process.  To 
verify  the  EC’s  functionality  a  similar  stepped 
approach  may  be  chosen: 


•  Initially,  static  inputs  are  provided  to  prove  the 
validity  of  the  static  recommendations. 

•  In  the  next  step  small  variations  of  single  and 
multiple  inputs  may  be  exercised.  It  is 
expected  that  the  advice  given  by  the  EC  is 
changing  also  only  slightly. 

•  Next,  dynamic  variations  should  be  performed. 

•  Finally,  stress  testing  with  a  high  workload  on 
the  EC  in  extremely  remote  operational 
scenarios  should  be  performed. 

Graceful  degradation  is  a  very  important  issue  of 
digital  flight  control  sytems  for  aircraft  with 
relaxed  or  without  stability.  Graceful  degradation 
simply  requires  that  the  system  should  not  rapidly 
change  the  functional  performance  level  in  case  of 
failures.  Because  with  full  operational 
performance  pilot’s  situational  awareness 
regarding  the  FCS  may  be  limited  due  to  other 
sophisticated  tasks  that  are  required  for  mission 
success.  Pilot  reaction  time  increases  and  failure 
transients  should  be  still  manageable. 

Considering  the  situational  awareness  problems 
with  modern  flight  decks  in  civil  transport  aircraft, 
graceful  degradation  has  a  wider  impact  than  on 
FCS  only.  Because  an  EC  will  lead  to  less  pilot 
workload  it  is  not  guaranteed  that  an  EC  will 
enhance  situational  awareness.  Prolonged  pilot 
reaction  times  together  with  failure  masking  may 
lead  to  hazardous  situations. 

The  problem  with  graceful  degradation 
requirements  is  the  impact  on  the  complete 
hardware  and  software  architecture.  However, 
developers  are  forced  to  mitigate  hazards 
identified  in  the  system  safety  process  in  a 
consistent  manner. 

Verification  of  graceful  degradation  requirements 
is  usually  performed  by  design  assessments.  The 
compliance  with  these  requirements  is  further 
investigated  during  integration  testing  on  all 
levels. 


3.3  Equipment 

In  most  instances  a  FCS  comprises  all  equipment 
necessary  to  perform  safety  critical  functions 
because  FCS  has  together  with  engine  control 
systems  and  armament  control  systems  the  highest 
criticality  of  all  aircraft  systems.  The  EC  instead 
is  not  a  stand  alone  system.  The  equipment  used 
for  information  gathering  from  the  pilot  may  be 
part  of  an  EC  system,  but  the  EC  relies  heavily  on 
the  data  provided  by  other  aircraft  systems 
designed  with  respect  to  varying  integrity  levels. 
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4. 


CONCLUSIONS 


The  engineering  tasks  related  to  equipment 
development  comprise  hardware  and  software 
development  and  several  supporting  processes. 
The  most  important  one  considers  safety  and  is 
highly  related  to  the  redundancy  management 
features  of  the  particular  system. 

Hardware  development  covers  sensors,  computers 
and  actuator  hardware.  For  FCS  as  well  as  for  EC 
well  established  methods  are  applicable  covering 
design  assessments,  environmental  qualification 
and  endurance  testing. 

Software  design  and  verification  has  usually  to  be 
performed  according  to  established  software 
standards.  However,  software  engineering  is  a 
rather  young  engineering  discipline  with  some 
generally  unsolved  substantial  shortfalls  in  the 
verification  process,  especially  if  systems  are 
complex  what  they  usually  are.  In  the  last  decade 
low  level  implementation  problems  were  mainly 
solved  by  improved  development  tools  and 
procedures.  On  this  level  complexity  should  not  be 
an  issue. 

It  can  be  assumed  that  verification  of  EC  software 
will  benefit  from  the  general  achievements  or  that 
new  methods  required  can  be  invented  with 
reasonable  effort. 

Verification  of  the  software  and  system  integration 
process  has  to  be  considered  in  more  depth 
because  unexpected  software  errors  are  mainly 
introduced  on  higher  integration  levels.  Because 
all  possible  test  cases  cannot  be  performed,  the 
system  architecture  should  seperate  functions  and 
minimize  the  interfaces  between  them  as  far  as 
possible.  Then  a  bottom-up  integration  test 
approach  shall  be  applied.  Test  cases  should  cover 
normal  operation  tests,  timing  tests  and  robustness 
tests.  Mode  changes  should  be  verified 
thoroughly. 

For  the  EC  special  emphasis  is  required  to  validity 
of  the  data  received  from  other  aircraft  systems. 
Integrity  shortfalls  of  single  aircraft  systems  have 
an  impact  on  the  EC  integrity  especially  when 
invalid  data  is  labeled  as  valid. 

To  minimize  the  effect  a  system  safety  programme 
should  be  performed  to  trace  possible  hazards  and 
to  mitigate  them  by  appropriate  means,  usually  by 
redundancy  management  considerations. 
Redundancy  may  also  be  required  due  to  the 
graceful  degradation  requirement. 


In  this  paper  two  EC  models  were  discussed. 
While  the  introduction  of  an  EC  as  supervisory 
controller  in  an  inhabited  aircraft  cannot  be 
recommended  today,  the  EC  as  adviser  promises 
functional  enhancements  that  have  a  potential  for 
increased  safety  and  survivability.  However,  the 
integrity  of  the  EC  cannot  be  better  than  of  the 
input  data  provided  by  other  aircraft  systems  and  a 
nearly  perfect  EC  may  be  a  source  of  pilot 
overconfidence  in  the  given  advice. 

A  route  to  certification  was  derived  from  common 
certification  practice  with  FCS.  This  initial 
analysis  has  revealed  no  general  and  unsolvable 
certification  problems.  However,  many  issues  have 
to  be  solved  in  particular  for  the  certification  of 
an  EC. 

Finally,  it  has  to  be  noted  that  the  EC  is  sensitive 
to  unexpected  failures  like  other  systems.  These 
unexpected  failures  are  mainly  caused  by  the 
slight  differences  of  intentions  and  written 
requirements  which  can  be  usually  observed  in 
case  of  complex  systems.  Furthermore,  the  EC 
provides  no  means  to  solve  this  problem  for  other 
aircraft  systems. 
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ABSTRACT 

Certification  can  be  defined  as  the 

“Procedure  by  which  a  third  party  gives  written 
assurance  that  a  product,  process  or  service 
conforms  to  specified  requirements.” 

(ISO/IEC  Guide  2). 

There  are  no  existing  standards  to  allow  the  certification  of 
avionics  resident  Knowledge  Based  Systems  (KBS),  as  may 
be  a  technical  part  of  the  future  aircraft  Electronic  Crewman 
(EC),  and  none  yet  devised  for  an  EC.  It  is  argued  that  the 
certification  of  KBS  technology  itself  requires  additional 
considerations  to  those  already  applied  through  software 
engineering  standards.  With  KBS  there  are  additional 
Verification  and  Validation  (V&V)  considerations  that  must 
be  placed  on  the  knowledge  base,  the  KBS  model,  and  the 
output  advice.  In  addition,  a  Man  Machine  System  (MMS) 
that  includes  an  EC  will  involve  a  cognitive  control  loop. 

This  loop  exists  because  the  EC  and  the  aircrew  will  be 
developed  as  a  team  sharing  flight  and  mission  related 
cognition.  Thus  the  homeostasis  of  the  cognitive  control  loop 
relies  on  a  sharing  of  cognitive  aspects  of  EC  /  operator 
teaming  including  the  team’s  complementary  application  of 
pertinent  mission  knowledge,  flight  information,  operational 
expertise,  and  situational  awareness.  It  is  argued  that  such  a 
loop  would  have  to  be  open  to  examination,  and  reliably 
perform  its  required  control  functions,  to  gain  acceptance  for 
its  adoption  and  certification  for  airborne  use.  Related  issues 
to  the  problem  area  are  discussed. 

BACKGROUND 

The  difficulties  involved  with  the  certification  of  a  system 
vary  with  the  system  application  and  the  complexity  of  its 
constituent  parts  (for  example,  its  software,  hardware,  and 
liveware).  The  certification  of  software  based  avionics  is 
based  on  well-established  V&V  principles  and  developing 
methods  such  as  employed  by  software  Static  Code  Analysis. 
Verification  is  the  process  of  evaluating  a  system  or 
component  to  determine  whether  the  products  of  a  given 
development  phase  satisfy  the  conditions  imposed  at  the  start 
of  the  phase  (Def  Stan  00-55).  In  contrast,  Validation  is  the 
evaluation  process  that  ensures  that  the  developed  product  is 
compliant  to  the  given  product  requirements. 

Hardware  principles  are  also  well  established.  System 
acceptance  principles  are  less  well  established  but  are  based 
on  specified  system  performance  criteria  and  a  regime  of 
testing,  evaluation  and  acceptance.  Aircrew  certification  is  in 
the  form  of  an  endorsement  of  their  reliability  and  expertise. 
This  endorsement  process  includes  an  appraisal  of  the 
standard  of  aircrew  ab  initio  &  continuation  training,  and 
frequent  in-flight  and  simulator  based  evaluations  of  their 


level  of  competence  with  relation  to  the  conduct  of  safe  flight 
and  associated  aircrew  tasks. 

Current  UK  Military  certification  of  Safety  Critical  Systems 
in  aircraft  involves  satisfaction  of  many  requirements,  much 
of  these  contained  in  the  UK  software  engineering  Defence 
Standard  (Def  Stan)  00-55,  Def  Stan  00-56,  &  Def  Stan 
00-58.  Applied  with  less  rigour,  these  standards  can  also  be 
applied  to  non-safety  critical  systems  such  as  mission  critical 
systems.  The  focus  for  the  certification  of  a  mission  critical 
system  is  the  criticality  of  that  system  performance  to  mission 
effectiveness.  In  contrast,  in  certifying  a  safety  critical 
system,  the  criticality  of  that  system  to  the  safe  performance 
of  the  mission  should  be  the  prime  consideration. 

However,  none  of  the  existing  standards  specifically 
addresses  the  certification  of  Knowledge  Based  Systems,  or 
an  EC,  or  an  EC  based  mission  or  safety  critical  system,  as 
might  be  part  of  the  future  aircraft  Man-Machine  System 
(MMS).  Such  systems  could  be  associated  with  the 
management  and  control  in  a  fighter  cockpit  or  with  the 
tactical  conduct  of  a  mission  in  the  rear  crew  area  of  a  multi 
crew  aircraft. 

Current  avionics  systems  provide  various  forms  of  feedback 
to  the  aircrew  on  system  performance,  its  operating  status, 
and  aircraft  progress  within  the  flying  environment.  Control 
of  the  aircraft  systems  is  generally  either  automatic  under  the 
supervision  of  the  aircrew,  or  actioned  directly  by  the  aircrew. 
Engineered  systems  and  aircrew  manifest  a  strict  partition  in 
system  control  functions.  However,  if  control  feedback 
changes  its  form  from  that  of  information  to  that  of  advice, 
then  system  control  will  be  truly  system  based  and  shared 
between  the  human  and  the  engineered  components  of  the 
system.  Note  that  this  does  not  mean  that  the  responsibility 
for  the  direction  of  the  system  is  a  system  function. 
Responsibility  for  system  direction  should  remain  within  the 
remit  of  designated  aircrew.  However,  the  control  of  the 
system  to  maintain  its  direction  towards  planned  system  goals 
becomes  the  control  task  of  a  ‘Joint  Cognitive  System’ 
(Hollnagel  &  Woods,  1984). 

Control  theory  has  long  been  capable  of  specifying  the 
dynamic  control  changes  required  to  direct  the  necessary 
machine  processes  to  adapt  to  changes  caused  by 
environmental  influences  or  operator  inputs  to  the  machine 
processes.  The  seminal  works  by  Weiner  (1948)  and  Ashby 
(1956)  on  cybernetics  introduced  the  idea  of  homeostasis  in 
MMS;  adaptive  control  to  maintain  a  stable  and  required  man- 
machine  system  state.  However,  a  MMS  that  includes  an  EC 
will  involve  a  control  loop  whose  homeostasis  relies  on  a 
sharing  of  cognitive  aspects  of  EC  /  operator  teaming 
including  the  team  application  of  pertinent  knowledge, 
expertise,  and  situation  awareness.  Such  a  loop  would  have  to 
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reliably  perform  its  required  control  functions  to  gain 
acceptance  for  its  adoption  or  certification:  in  other  words  it 
must  do  what  it  is  required  to  do  in  a  variety  of  situations  and 
in  a  sufficiently  consistent  manner  to  allow  confidence  in  its 
acceptance. 

Further  to  this  discussion,  EC  related  certification  should 
involve  V&V  examination  of  at  least  four  separate  processes 
as  associated  with: 

•  EC  Model  certification; 

•  Certification  of  the  elicited  inputs  for  the 
EC; 

•  EC  software  certification 

•  MMS  System  certification. 

The  first  process  can  arguably  be  catered  for  by  logical  model 
generation,  including  tests  for  validity  and  for  good  result 
replication.  The  second  process  would  probably  require 
iterative  examination  of  the  knowledge,  rule  (possibly  skill) 
inputs  by  several  appointed  and  independent  assessors,  these 
assessors  having  a  roles  similar  to  those  of  present  day  flight 
crew  assessors.  The  certification  through  such  a  process 
might  result  in  an  agreed  rating  of  the  EC  depending  on  its 
performance  and  maturity.  The  third  and  last  processes  can 
be  catered  for  by  current  certification  methods  -,  does  the 
system  reliably  and  validly  meet  the  requirements  of  the 
system  specification  and  allow  verification  of  its  quality  of 
build?  The  last  process  would  encompass  the  results  of  the 
previous  processes  and  might  also  include  an  assessment  of 
the  competence  level  of  the  EC  manufacturer  as  suggested  by 
the  current  Software  Engineering  Institute  (SEI)  1996 
Capability  Maturity  Model  (CMM). 

To  be  operationally  effective  an  EC  would  have  to  be 
continually  developed  (trained).  Changes  in  any  one  of  the 
first  three  of  the  above  processes  could  affect  the  veracity  of 
the  whole  EC  /  operator  team  as  might  a  change  of  operator. 
Therefore,  the  examination  of  all  of  the  above  processes 
would  need  to  be  iterative  and  the  findings  applied  as 
refinements  to  the  MMS  throughout  the  EC  life  cycle.  The 
amount  of  iteration  and  refinement  allowed  by  the 
certification  process  would  have  to  be  carefully  considered 
and  administered  (for  example  the  problems  of  configuration 
control). 

Further,  it  is  to  be  expected  that  eventually  with  EC  design, 
since  an  operator’s  cognitive  control  performance  varies 
between  and  during  a  flight  and  also  with  changing  flight 
conditions,  that  the  EC  could  adapt  to  changes  in  its 
environment.  The  EC  would  have  to  be  aware  of  such 
changes  and  be  able  to  adapt  its  teaming  assistance 


accordingly.  Thus,  the  concept  of  a  future  adaptive  controller 
must  cover  a  team  interaction  policy  as  well  as  a  man- 
machine  system  control  policy  (Hollnagel,  1995)  -,  and  all 
this  detail  has  to  be  unambiguously  specified  during  concept 
and  development  prior  to  the  EC  build  and  eventual 
certification. 

Note  that  the  EC  consideration  in  this  article  refers  to  a  ‘one- 
to-one’  pairing  between  a  single  member  of  aircrew  and  a 
single  EC.  The  certified  use  of  any  EC  technology  within  a 
multi  crewed  aircraft  environment  would  present  additional 
unique  and  complex  teaming  problems. 

PURPOSE 

The  remainder  of  the  paper  will  amplify  some  of  the 
arguments  and  issues  related  to  the  problems  of  the 
certification  of  a  MMS  cognitive  control  loop.  This 
amplification  considering  the  performance  requirements  of 
the  whole  EC  /  operator  team,  and  the  affects  on  the  processes 
of  control  of  diverse,  hostile,  and  uncertain  environments 
(MacLeod  &  Wells,  1997). 

In  addition,  the  need  for  a  method  supporting  the  specification 
of  system  cognitive  functions  will  be  briefly  visited  to 
introduce  an  issue  that  it  is  argued  must  be  addressed  to  allow 
EC  certification  within  a  system  (MacLeod  &  Scaife,  1997). 
System  cognitive  control  relies  on  an  imderstanding  of  what 
system  functionality  is  necessary  to  support  that  level  of 
system  control  required  to  support  system  direction  towards 
the  fulfillment  of  system  goals  and  the  effective  satisfaction 
of  system  purpose. 

Figure  One  represents  a  model  of  process  control  for  a 
military  aircraft  system.  Examination  of  the  figure  shows  that 
the  problem  space  of  EC  work  would  have  to  reside  to  a 
greater  or  lesser  extent  across  three  areas  of  the  model, 
namely: 

•  Sensor  control; 

•  Cognitive  control; 

•  Equipment,  aircraft,  &  weapons  control. 

With  an  MMS  including  an  EC,  situational  awareness  must  be 
shared  through  some  form  of  dynamic  apportionment  within 
the  team  (MacLeod,  Taylor,  Davies,  1995).  Further,  the 
model  deals  with  uncertainty  as  might  exist  in  a  hostile 
military  environment  and  indicates  how  that  uncertainty  may 
be  caused  and  its  affects  ameliorated  depending  on  the 
applied  level  of  aircrew  expertise.  Thus  both  expertise  and 
planning  /  tasking  mediate  the  cognitive  loop  appraising  the 
environment. 
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Figure  One:  A  simple  aircraft  process  control  model  considering  a  single  operator 
(Acknowledgments  to  Neisser,  U.,  1976  and  Hollnagel,  E.,  St  Lary  Lecture  Notes,  1995) 


This  loop  is  susceptible  to  artifacts  of  uncertainty  that  can 
evoke  unplanned  observations.  These  unplanned 
observations  can  be  detrimental  to  MMS  control  if  outside 
the  scope  of  applied  expertise. 

ISSUES 

The  EC  specific  issues  that  will  be  briefly  discussed  are 
listed  below.  Many  of  the  issues  are  strongly  interrelated. 
For  example,  the  form  of  criticality  of  the  system  will 
dictate  the  level  of  V&V  attention  to  Safety  whether  the 
system  is  considered  to  be  Safety  Critical  or  Mission 
Critical.  The  listed  issues  are  not  considered  to  be  an  all- 
encompassing  set  but,  nevertheless,  are  sufficiently 
representative  of  the  problems  associated  with  the  subject 
certification  process  to  support  a  discussion  of  the  problem 
space,  this  space  introduced  in  the  next  section. 

♦  Safety; 

♦  form  of  criticality; 

♦  maturity  of  model; 

♦  methodology; 

♦  life  cycle; 

♦  V&V  methods; 

♦  testing; 

♦  maintenance  &  refinement; 

♦  stand-alone  Vs  real-time  embedded  distributed 
systems. 

CONSIDERATION  OF  PROBLEM  SPACE 

The  problem  space  encompassing  current  concept,  design 
and  development  of  a  complex  system  can  be  practically 
defined  by  various  existing  methods. 


Taking  as  an  example  the  problems  associated  with  the 
certification  of  avionics  based  KBS  systems  (subject  of  a 
current  study  being  undertaken  by  Aerosystems  on  behalf 
of  DERA  Famborough,  Air  Systems  Sector  [H  Howells]) 
which  are  likely  to  form  part  of  an  EC.  Excellent  standards 
such  as  the  ESA  PSS-05  are  not  readily  applicable  to  KBS 
as  they  were  devised  in  the  period  of  3^**  Generation 
Languages  (GL)  when  the  ‘Waterfall’  model  was 
predominant.  This  model  represents  the  software 
development  process  as  serid  in  nature  through  stages  of 
Design,  Code,  and  Test. 

The  iteration  required  in  the  specification  and  production  of 
KBS  makes  the  use  of  the  rigid  specification  requirements 
and  serial  nature  of  the  ‘Waterfall’  model  questionable  as  a 
template  on  which  to  base  certification  of  KBS.  Thus  the 
PSS-05  model  would  have  to  be  tailored  and  refined  if  it 
was  to  have  any  applicability  to  the  subject  area.  Attempts 
to  tailor  the  ESA  standard  for  use  in  the  V&V  of  KBS 
include  the  ViVa  Model  (Schlee  &  Lackingen,  1995).  In 
addition,  the  usability  of  the  UK  Def  Stan  00-55  has  been 
approached  by  researchers  consideration  the  applicability 
of  the  development  and  use  of  formal  methods  for  use  in 
Knowledge  Engineering,  methods  now  well  integrated  into 
good  current  SE  practices  especially  with  respect  to  safety 
critical  systems  (van  Harmelen,  1995).  For  the  sake  of  this 
article,  the  likely  problem  space  for  certification  of  a  KBS  / 
EC  will  be  considered  under: 

1.  The  utility  of  existing  standards  and  methods. 

2.  The  complexity  and  number  of  issues  involved  in  EC 
system  certification. 

3.  T^e  overall  system  architecture  in  which  the  EC 
resides. 
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The  utility  of  existing  standards  and  methods 

Existing  standards  and  methods  are  designed  to  be  applied 
to  current  Software  Engineering  (SE)  practices  (for 
example  Def  Stan  00-55,  ESA  PSS-05)  and  Systems 
Design  practices  (for  example  lEE  PI 220,  ARP  4754). 
Current  software  standards  are  naturally  products  of  their 
time  and  would  all  require  tailoring  to  fit  the  specific 
problem  area  of  future  EC  certification 

Conventional  SE  recognizes  the  usefulness  of  both  formal 
and  informal  methods.  For  example,  informal  high  level 
specification  can  be  performed  by  structural  analysis 
method  such  as  Yourdon;  formal  methods  such  as  VDM 
allow  the  detailed  specification  of  the  software 
functionality;  and  results  can  be  evaluated  as  prototypes  by 
executable  specifications  such  as  PAISLey.  In  contrast, 
though  it  is  early  days  yet,  it  is  likely  that  effective  SE 
methods  applicable  to  KBS  will  be  a  mixture  of  formal  and 
informal  methods  such  as  MIKE  (Angele,  Fensel,  Landes, 
Neubert,  &  Studer,  1993). 

Showing  their  recent  maturity,  system  engineering  models 
emphasize  early  requirements  capture  and  an  iterative 
approach  to  the  specification  of  system  fiinctions,  the 
separate  consideration  of  logical  and  physical  models, 
traceability  of  requirements,  iteration,  trade-off  studies,  and 
the  need  for  careful  control  over  the  overall  engineering 
process.  A  simple  system-engineering  model  is  shown  in 
Figure  Two  overleaf. 

The  traceability,  control  and  iterative  nature  of  the  systems 
engineering  design  instinctively  seems  to  be  more 
applicable  to  the  development  of  the  future  EC,  this 
extrapolating  from  current  considerations  on  KBS  issues. 
However,  the  strong  emphasis  on  the  early  capture  of 
system  requirements  might  require  some  amelioration  with 


respect  to  the  EC  component  of  the  overall  MMS.  Further, 
the  spectrum  of  EC  consideration  (see  Figure  One)  should 
require  that  the  specification  of  the  system,  and  the  EC, 
contain  not  only  a  logical  consideration  on  the  engineered 
system  but  also  on  the  cognitive  requirements  of  that 
system.  For  example  requirements  on  understanding, 
anticipation,  control,  situation  analysis,  supervision  -  to 
name  but  a  few  (MacLeod  et  al,  op  cit.).  Importantly, 
cognitive  requirements  are  necessary  for  the  build  of  an  EC 
as  part  of  the  necessary  specification  of  its  support  role  to 
the  direction  and  control  of  a  military  aircraft  system.  This 
role  will  require  that  complementary  advice  to  the  aircrew 
includes  suitable  elements  of  anticipation  and  prediction  on 
system  performance  under  environmental  uncertainty. 

Existing  standards  and  methods  can  give  much  guidance  to 
the  route  needed  to  satisfy  future  EC  V&V  requirements, 
requirements  that  will  inevitably  be  tailored  to  meet  the 
needs  of  the  EC  certification  process.  All  of  the  above 
arguments  are  affected  by  the  complexity  of  the  issues 
affecting  the  V&V  process  associated  with  an  EC.  These 
issues  will  be  discussed  in  the  next  section. 

The  complexity  and  number  of  issues  involved  in  EC 
system  certification 

There  are  many  interrelated  issues  involved  in  the 
certification  of  an  EC.  The  seeming  complexity  of  EC 
related  issues,  compared  to  those  applicable  to  current 
engineered  systems,  is  probably  as  much  related  to  the 
relative  newness  and  lack  of  understanding  of  the  subject 
area  as  to  their  actual  complexity.  The  number  of  issues 
that  could  be  discussed  is  indicated  by  the  contents  of  the 
list  in  the  Issues  Section  above.  Considering  that  list  as 
representative  of  the  issues  to  be  addressed,  these  listed 
issues  will  be  discussed  in  the  following  paragraphs. 


Figure  2:  Example  of  a  generic  SE  process  model  (after  lEE  P1220) 
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Safety 

The  Safety  Issues  in  the  design  of  an  EC  will  depend  on  the 
implementation  methods  chosen  for  the  EC,  the  form  of 
criticality  of  the  system,  and  the  maturity  of  the  safety 
analysis  techniques  used  in  verification  and  to  assess  the 
system  for  certification.  It  is  suggested  that  whatever 
integral  KBS  system  is  used,  and  depending  on  the  form  of 
the  other  components  incorporated  into  the  EC  sub  system, 
the  EC  will  have  to  be  seen  to  be  designed  as  ‘fit-for- 
purpose’  with  the  use  of  generic  components  (for  example, 
COTS)  being  carefully  argued  as  being  context 
independent  or  dependent  depending  on  their  purpose  in  the 
EC. 

Form  of  criticality 

The  importance  of  the  form  of  criticality  has  already  been 
stressed.  Safety  issues  will  vary  with  the  form  of  criticality 
as  will  their  examination.  For  example,  the  safety  analysis 
techniques  applied  for  a  Safety  Critical  System  would  have 
to  be  applied  with  greater  rigour,  or  allow  greater 
confidence  to  be  placed  on  their  results,  than  similar 
techniques  applied  to  a  Mission  Critical  System. 

Maturity  of  model 

Maturity  of  the  EC  the  model  can  be  expressed  in  terms  of 
time  in  service,  proven  usefulness  and  usability,  and 
reliability  in  performance.  Also  important  is  consideration 
on  the  maturity  of  the  technology  and  techniques  used  to 
specify  and  develop  the  EC.  The  more  mature  the  model 
the  easier  should  be  the  certification  process. 

Methodology 

Issues  of  methodology  are  closely  associated  with  the 
issues  of  criticality  and  maturity.  Certification  will  be 
difficult  without  validated  methods  of  assessing  the  build, 
knowledge,  and  advice  /  competence  of  the  EC.  These 
methods  will  have  to  be  both  qualitative  and  quantitative  in 
nature.  With  the  qualitative  methods,  argued  as  appropriate 
to  assessing  the  HOW  and  WHY  aspects  of  work 
knowledge  and  advice,  there  will  need  to  be  developments 
in  the  assessment  of  the  accuracy  and  validity  of  results  (for 
example,  method  such  as  ‘triangulation’  -.  Richardson, 
1996). 

Life  cycle 

There  are  many  definitions  of  a  design  life  cycle,  these 
varying  between  professions  and  applications.  Sufficient  to 
say  that  the  software  engineering  consideration  of  a  system 
development  life  cycle  span  is  generally  shorter  than  that 
proposed  by  systems  engineering,  which  again  is  generally 
shorter  than  that  envisaged  by  the  customer  of  the  system. 
Life  cycle  considerations,  and  the  frequency  of  system 
upgrades,  have  implications  with  relation  to  considerations 
on  the  maturity  of  the  EC  and  will  be  discussed  in  greater 
detail  when  considering  the  Maintenance  and  Refinement 
of  the  EC. 

V&V  methods 

V  8lW  methods  for  an  EC  will  be  difficult  to  apply  because 
of  the  diverse  technological  developments  that  are  liable  to 
be  involved  in  its  build  (for  example,  neural  nets,  KBS, 
relational  data  bases,  automatic  communications).  Existing 


approaches  to  KBS  V&  V  lack  maturity  and  are  too  narrow 
in  scope  to  meet  current  certification  requirements  for 
airborne  avionics  systems.  The  KBS  V&V  approaches 
tend  to  focus  on  the  KBS  and  ignore  the  real  world  airborne 
issues  associated  with  the  use  of  KBS  within  an  aircraft 
system,  issues  that  must  be  considered  by  the  certification 
process.  Current  SE  practices  and  methods  are  available  to 
allow  software  build  to  meet  the  challenges  of  the  existing 
certification  processes.  KBS  and  the  EC  should  consider 
the  best  practices  existing  with  the  current  SE  profession, 
including  the  ’Old  Physical  Model’,  that  are  applicable  to 
their  certification  process.  Further,  consideration  of  the  old 
model  should  allow  the  existing  expectations  of  the 
certification  community  to  be  understood  in  order  that  the 
similarities  and  differences  of  the  new  application  of 
technology,  through  the  EC,  can  be  argued  and  understood 
by  that  community.  It  is  suggested  that  it  is  only  by  this 
route  that  the  certification  community  can  be  convinced 
that  processes  can  be  set  in  place  allowing  the  certification 
of  airborne  KBS  /  EC. 

Testing 

System  testing  requires  a  good  knowledge  of  what  is  to  be 
tested,  how  it  is  to  be  tested,  and  the  expected  system 
performance.  Models  of  the  system  and  of  its  constituent 
parts  can  depict  such  knowledge.  Progressive  testing  of  a 
system  is  the  feed  to  the  V&V  process  that  gives  assurance 
that  the  system  design  is  progressing  as  planned  and  assists 
in  de-risking  the  design  by  highlighting  inherent  problems. 

A  system  model  is  traditionally  rooted  in  system 
requirements.  A  requirements  specification  is  argued  to  be 
just  as  essential  for  an  EC  as  for  any  other  engineered 
system.  However,  the  associated  functional  specification 
of  the  EC  will  have  to  be  constructed  iteratively  in  the  light 
of  the  designers’  increased  understanding  of  problems  and 
processes  associated  with  the  EC  work  area.  Indeed,  many 
of  the  EC  functions  will  be  evolve  from  progressive  testing. 
Therefore,  the  waterfall  model  is  not  appropriate  as  a  more 
iterative  approach  will  be  needed  to  EC  design,  and 
probably  to  the  design  of  most  future  complex  systems. 
Aspects  of  the  Systems  Engineering  model  (see  Figure  2) 
are  more  appropriate  than  the  standard  waterfall  model. 

However,  it  is  probable  that  the  EC  development  model 
will  be  different  from  the  systems  engineering  model  but 
will  have  to  interlace  with  that  model  at  different  stages  of 
the  overall  system  design  process.  Testing  of  the  EC  must 
rely  on  a  good  knowledge  of  the  anticipated  utility  of  the 
models  associated  with  the  build,  models  that  will  be 
refined  in  the  light  of  test  results  and  design  experience. 
Integration  of  sub  systems  such  as  an  EC  into  the  overall 
system  architecture  will  be  important.  The  models  of  real 
time  sub  systems  and  systems  must  be  compatible  and  their 
development  plans  must  be  carefully  integrated.  Therefore, 
for  an  EC  these  models  should  encompass  as  a  minimum: 

•  An  EC  Model; 

•  A  model  of  elicited  knowledge  (Use  of  this  model  will 
vary  depending  on  the  needs  of  the  technology.  For 
example  there  are  differences  in  the  use  of  knowledge 
between  rule  and  case  based  KBS); 

•  An  EC  /  Aircrew  teaming  model; 
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•  An  aircraft  system  architecture  model; 

•  An  aircraft  system  function  model; 

•  An  aircraft  system  performance  model. 

Maintenance  &  refinement 

A  simple  definition  of  a  design  life  cycle  is  ‘firom  birth  to 
death’  of  the  system.  However,  it  can  be  argued  that  such  a 
definition  is  incorrect  as  even  when  a  system  is  ‘retired’  the 
lessons  learnt  from  that  system  will  affect  the  birth  of 
another  the  model  is  in  reality  one  approaching  perpetual 
motion.  It  is  suggested  that  whatever  technology  and 
disciplines  support  the  construction  of  an  EC,  die 
continuous  nature  of  system  maintenance  and  refinement 
that  an  EC  will  need,  to  promote  and  maintain  its 
efficiency,  fits  it  better  into  the  a  ‘perpetual  motion’  form 
of  life  cycle.  To  maintain  such  momentum  requires  that 
testing  is  conducted  that  is  associated  with  effective  V&V 
methods  and  an  engineering  model  of  the  system  that 
allows  progressive  refinement  and  an  associated  evolving 
EC  system  maturity. 

The  overall  system  architecture  in  which  the  EC 
resides 

Current  KBS  concepts  are  developed  on  systems  in  ‘stand¬ 
alone’  environments  that  are  divorced  fi-om  the 
complexities  of  aircraft  systems  (though  simulations  of 
some  of  such  systems  may  be  used  to  assist  the 
development).  In  reality,  the  future  KBS  and  EC  will  have 
to  exist  within  a  real-time  distributed  system  of  some  form. 
To  be  accepted  by  the  crew  the  EC  must  be  situationally 
aware  and  be  useful  if  not  invaluable.  To  be  useful  as  a 
surrogate  crewman  the  EC  contributions  to  the  work  of  the 
man-machine  team  must  be  timely,  appropriate,  and  the 
quality  of  advice  must  be  consistent. 

Timeliness  is  a  serious  inherent  problem  that  has  not  been 
fiilly  addressed  by  current  KBS  developments  but  that  must 
be  resolved  with  relation  to  in-service  use  of  KBS  and  the 
future  EC.  In  current  real-time  systems  correct  answers 
must  be  produced  within  strict  time  constraints.  EC  advice 
will  have  different  requirements,  with  relation  to  time 
constraints  and  output  accuracies  /  quality,  than  a  system 
that  merely  calculates  and  presents  feedback  data  or 
information.  Nevertheless,  certification  processes  will 
almost  certainly  demand  that  the  bounds  or  tolerances  of 
the  EC  performance  are  known  and  that  these  bounds  do 
not  detract  from  overall  system  performance  or  its  safe  use. 
Possibly  two  KBS  will  be  needed  as  a  check  on  one 
another;  number  one  producing  the  best  solution  possible 
within  the  allowed  time  fi-ame  and  the  other  producing  an 
‘optimum’  solution.  This  optimum  solution  may  or  may 
not  fit  the  required  time  frame,  and  might  only  be  made 
available  to  the  operator  on  request,  but  could  be 


engineered  as  a  check  on  the  performance  quality  of  the 
number  one  KBS. 

To  conclude  the  discussion  in  this  section.  Table  One 
overleaf  lists  the  main  issues  raised  by  the  Def  Stan  00-55 
with  relation  to  the  acceptance  of  avionics  based  Safety 
Critical  Software.  With  less  rigour  in  application  these 
practices  can  also  be  applied  to  the  certification  of  Mission 
Critical  Software.  The  table  considers  these  issues  against 
the  development  of  a  typical  stand-alone  KBS  development 
to  illustrate  the  differences  between  the  current  practices  on 
SE  certification  and  the  development  practices  for  KBS. 
Please  note  that  these  differences  are  presented  solely  to 
give  an  illustration  of  the  possible  extent  of  the 
development  of  EC  design  practices  that  are  needed  to  meet 
future  SE  practices.  In  reality,  the  particular  differences 
between  the  needed  EC  practices  to  achieve  certification, 
and  current  SE  practices,  will  be  dictated  by  the  nature  of 
the  technologies  used  for  the  EC  build  and  could  be  very 
different  to  those  presented  in  Table  One. 

CONCLUSION 

There  is  a  great  deal  of  practical  consideration  that  needs  to 
be  placed  on  the  certification  of  the  EC.  This  consideration 
starting  from  the  considerations  of  present  V&V  practices 
for  KBS,  the  gap  between  KBS  V&V  and  current  SE  and 
system  certification  requirements,  and  continuing  to  the 
much  greater  issues  involved  with  the  certification  of  an 
EC. 

The  current  requirements  for  avionics  and  aircraft 
certification  are  well  known.  Starting  from  the  baseline  of 
the  current  requirements,  we  need  to  practically 
complement  and  supplement  these  requirements  to  meet  the 
certification  case  for  the  EC  and  its  supporting 
technologies.  It  is  important  that  a  practical  engineering 
emphasis  is  maintained,  drawing  from  academic  and 
research  studies  in  the  field,  as  the  EC  certification  process 
will  probably  be  similar  to  those  employed  currently. 
Current  certification  processes  require  more  than  a  method 
of  conducting  the  certification,  the  process  also  requires 
real  time  system  operation,  system  evaluation,  and  the 
production  of  sufficient  quality  evidence  to  support  the 
independent  certification  of  the  subject  system. 

We  are  emerging  from  an  era  where  the  overall  emphasis 
was  on  developing  technology  and  methods  to  suite  the 
concept  of  the  EC.  This  work  must  not  cease.  However,  it 
is  time  to  change  some  of  the  emphasis  of  the  work  towards 
a  better  consideration  on  the  practical  real  time  applications 
of  an  EC  and  its  quality  introduction  into  the  operational 
world  as  part  of  avionics  systems.  This  task  is  as 
challenging  as  any  that  have  gone  before. 
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Def  Stan  00-55  Section 

ICovered  by  Not  Covered 

SE  Practices  By  KBS 

Safety  Management 

Roles  and  Responsibilities 

✓ 

X 

Planning  &  Management  Processes 

✓ 

Quality  Assurance 

X 

Documentation 

X 

Development  Planning 

X 

Project  Risk  Analysis . 

X 

Verification  &  Validation  Planning 

X 

Configuration  Management 

✓ 

Selection  of  Methods 

✓ 

Code  of  Design  Practice 

X 

Selection  of  Language 

Selection  of  Tools 

Use  of  Existing  Software 

✓ 

Use  of  Diverse  Software 

X 

So|^are  Engineering  Processes  . 

. . . . . 

Development  Principles 

X 

Software  Requirement  Specification 

X 

Specification  Process 

Design  Process 

X 

Coding  Process 

Testing  and  Integration 

✓ 

Acceptance,  Certification  and  In-Service  Use 

Certification 

✓ 

Acceptance 

✓ 

. Replication _ _ _  _ _ _ _ _ 

X 

User  Instruction 

X 

In-Service 

Table  One:  Def  Stan  00-55  Considered  Issues  in  Certification  Compared  with  Current  KBS  Practices 
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1.  SUMMARY 

This  paper  describes  the  prototype  Approach  Procedures 
Expert  System  (APES),  developed  at  the  Vehicle^Pilot 
Integration  Branch  of  Wright  Laboratory  at  Wright- 
Patterson  Air  Force  Base.  Specifically,  the  paper 
describes  the  types  of  advice  provided  by  the  APES, 
discusses  the  processes  involved  with  its  implementation, 
and  presents  a  summary  of  the  results  obtained  in  a 
pilot-in-the  loop  evaluation  of  the  APES.  The  APES  is 
based  on  process  flow  models  that  capture  the  heuristics 
and  procedures  that  define  pilot  processes  in  performing 
the  various  tasks  of  an  instrument  approach.  The  pilot- 
in-the  loop  evaluation  results  suggest  that  APES  can 
improve  pilot  performance,  increase  situational 
awareness,  and  r^uce  workload,  especially  in  a  high 
workload  environment.  The  evaluation  also  indicated 
some  refinements  to  the  APES  algorithm  and  pilot- 
vehicle  interface. 

2.  INTRODUCTION 

The  number  of  procedures,  “rules  of  thumb”  and 
extensive  cognitive  demands  placed  upon  the  pilot  make 
the  instrument  approach  well-suited  for  a  decision  aid 
application.  The  pilot  not  only  must  recall  and  apply 
specific  instrument  flight  rules,  remember  correct  task 
sequences,  and  calculate  timings,  but  must  also 
accomplish  these  tasks  while  simultaneously  controlling 
the  aircraft  and  continually  monitoring  its  performance. 
In  addition,  the  pilot  may  need  to  hurriedly  replan 
according  to  air  traffic  controFs  (ATC’s)  redirection, 
requiring  the  pilot  to  reassess  approach  chart  data, 
correlate  that  data  with  current  aircraft  position,  and 
possibly  adjust  instrument  command  settings.  When 
performed  under  adverse  weather  conditions  with 
unfamiliar  approaches,  the  potential  for  pilot  error  can 
dramatically  increase. 

The  intent  of  the  APES  prototype  is  to  reduce  pilot 
workload  and  increase  situational  awareness  by 
providing  the  appropriate  advice  to  the  pilot  whenever  it 
is  most  needed  during  a  particular  phase  of  the  approach. 
The  overall  goal  is  to  mitigate  the  number  of  accidents 
and  incidents  that  are  attributed  to  pilot  error.  An 
example  of  where  APES  may  have  benefited  pilots  is 
with  the  recent  airline  accident  in  Guam  (Korean  Air 
Flight  801).  Preliminary  reports  indicate  that  the  pilots 
apparently  lost  situational  awareness  when  performing  a 
non-precision  approach,  thought  they  were  closer  to  the 
Guam  airport  than  what  they  really  were,  and  crashed 
into  the  terrain  killing  225  passengers.  It  is  hoped  that 


such  accidents  as  the  one  in  Guam  can  be  averted  if 
pilots  are  supported  with  a  decision-aid  such  as  APES  to 
assist  in  the  approach. 

3.  APES  DESCRIPTION 

The  APES  simultaneously  monitors  aircraft 
performance,  informs  the  pilot  of  appropriate  corrective 
actions  when  deviations  occur,  and  provides  procedural 
advice  according  to  the  phase  of  the  approach  (i.e., 
holding,  initial  approach,  final  approach,  missed 
approach).  In  doing  this,  APES  functions  in  two 
assistant  roles:  as  an  "advisory  copilot"  and  as  an 
"advisory  pilot."  As  an  "advisory  copilot,"  the  APES 
advises  and  prompts  the  pilot  as  a  copilot  would  in  a 
crew  environment,  such  as  advising  when  the  aircraft 
deviated  from  assigned  parameters  (e.g.,  altitude, 
airspeed,  etc.).  As  an  "advisory  pilot,"  the  APES 
provides  guidance  relevant  to  the  instrument  flight  rules 
(IFRs)  needed  for  the  specific  phases  of  the  2q)proach. 
Audio  is  the  primary  pilot-vehicle  interface  for  the 
APES.  Visual  messages  are  employed  for  redundancy 
and  when  the  use  of  audio  is  impractical. 

The  basic  architecture  of  the  APES  is  shown  in  Figure  I. 


Figure  1.  APES  Architecture 

As  depicted,  inputs  to  the  APES  are  derived  from  three 
sources:  a  set  of  inputs  describing  current  aircraft  flight 
parameters,  a  database  of  aircraft-specific  facts  and  a 
database  of  approach-specific  facts.  Most  of  the  inputs 
to  the  APES  are  provided  dynamically  by  the  host 
system.  Examples  of  the  type  of  inputs  that  is  used  by 
APES  include  the  following: 

•  Aircraft  Status  Data 

Current  Altitude  /  Heading  /  Airspeed 
Current  Navigation  Aid  Radial 

•  Aircraft-Specific  Facts 

Holding  /  Approach  Airspeed 
Fuel  Weight 


•  Approach-Specific  Facts 

Final  Approach  Course 

Minimum  Descent  Altitude  /  Decision  Height 

An  aircraft-specific  input  file  is  created  for  each  aircraft 
type  in  order  to  allow  a  generic  APES  to  be  embedded  in 
aircraft  (or  aircraft  simulators)  of  different  types.  An 
approach-specific  input  file  is  created  for  each  approach 
type.  This  file  is  based  on  approach-specific  process  flow 
charts  that  identify  approach  plate  parameters  and 
generic  pilot  tasks  and  decisions  that  are  required  during 
a  particular  phase  of  an  approach.  An  example  of  such  a 
process  flow  chart  is  illustrated  in  Figure  2. 

4.  APES  DEVELOPMENT 

4.1  Knowledge  Acquisition  and  Process  Flow 

The  first  step  in  the  development  of  the  APES  is 
capturing  the  expertise  of  an  experienced  pilot  through  a 
knowledge  acquisition  process.  This  process  involves 
conducting  extensive  iterative  interviews  with  pilot 
subject  matter  experts  to  identify  the  precise  steps 
required  for  flying  the  various  phases  of  an  instrument 
approach.  The  knowledge  engineer  then  models  the 
actions  recommended  by  the  expert  pilot  and  creates 
process  flow  diagrams  that  represent  the  general 
sequence  of  tasks  and  decisions  for  flying  instrument 
approaches. 

The  collection  of  process  flow  diagrams  provide  the 
foundation  for  ^gorithmic  development  to  be 
implemented  in  the  APES  prototype.  A  representative 
example  of  the  content  of  these  process  flow  diagrams  is 
shown  in  Figure  2.  In  this  chart,  the  process  flow  for  an 
approach-specific  process  is  depicted.  This  process 
defines  the  sequence  of  pilot  tasks  and  parameters 
specific  to  an  approach.  These  parameters  are  currently 
depicted  in  paper  ^proach  plates. 
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Rgure2.  Approach-Specific  Process  Flow  for 
the  Holding  Phase 


4.2  APES  Implementation 

The  knowledge  acquisition  process  is  also  used  to 
develop  a  control  scheme  diat  enforces  the  proper 
sequencing  of  rules.  The  expert  systems  development 
tool  used  for  APES  is  the  C  Language  Integrated 
Production  System  (CLIPS)  developed  at  the  NASA 
Johnson  Space  Center  (CLIPS,  1993).  The  CLIPS 
inference  engine  uses  the  Rete  algorithm  to  activate  a 
rule  whenever  one  or  more  of  its  antecedents  has 
changed.  Each  APES  rule  states  the  precise  conditions 
(antecedents)  that  must  be  satisfied  before  that  rule  is 
executed. 

The  APES  prototype  is  embedded  in  the  Microprocessor 
Applications  for  Graphics  and  Interactive 
Communications  (MAGIC)  simulator  at  Wright 
Laboratory  Vehicle-Pilot  Integration  Branch.  The 
MAGIC  simulation  is  driven  by  a  generic  F-16  aero 
model.  The  MAGIC  cockpit  control  and  display  suite 
used  for  development  of  the  APES  prototype  is  depicted 
in  Figure  3. 

4.2.1  Electronic  Approach  Plate  Formats 

The  APES  was  implemented  in  conjunction  with  the 
Electronic  Approach  Plate  (EAP).  TTie  approach  plate 


Figure  3.  MAGIC/APES  Cockpit 


is  a  map  the  identifies  all  the  relevant  parameters  (e.g., 
radio  frequencies,  course,  altitudes)  for  flying  the  various 
phases  of  a  specific  approach.  EAPS  are  electronic 
depictions  of  the  paper  approach  charts.  The  EAPs  are 
displayed  to  the  left  of  the  head  -  down  flight  display 
in  the  MAGIC  cockpit  (see  Figure  3).  Figure  4  shows 
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Figure  4.  Representative  Electronic  Approach  Plate  Pormat 


an  example  of  the  electronic  approach  plate  format  that 
was  used  in  the  pilot-in-the-loop  evaluation  of  APES. 

4.2.2  APES  Pilot-Vehicle  Interface 

The  primary  pilot  interface  for  the  APES  is  auditory.  To 
output  a  voice  message,  the  APES  passes  a  text  string  to 
the  voice  module  of  the  host  system,  which  is  responsible 
for  generating  the  corresponding  audio  message.  The 
*  voice  module  generates  appropriate  phrases  by 
combining  words  listed  in  a  vocabulary  database  of 
approximately  50  words. 

APES  voice  messages  arc  reinforced  with  textual  output. 
The  APES  continually  displays  updated  target  values  for 
radio  channel/  frequency,  altitude,  airspeed,  heading,  and 


course  in  a  scratchpad  area  to  allow  appropriate 
command  and  course  settings.  APES  also  displays 
command  settings  for  radio,  altitude,  airspeed,  heading 
and  course  on  a  dedicated  CRT. 

Table  1  depicts  the  various  types  of  deviation  advice 
given  to  the  pilot,  which  is  part  of  the  APES  “advisory 
copilot”  component.  As  shown,  this  advice  is  given  for 
the  following  parameters:  airspeed,  altitude,  heading  and 
course.  APES  advises  the  pilot  of  a  deviation  and  gives 
correction  whenever  the  pilot  deviates  outside  a 
parameter’s  tolerance  window  set  by  APES.  For 
example,  if  the  pilot  deviates  more  than  +/-  2  degrees 
from  course,  the  APES  “advisory  copilot”  component 
announces  a  new  heading  and  turn  direction  to  re- 


AIRSPEED 

“Maintain  XXX  knots” 

Desired  airspeed  displayed  in  the  scratch¬ 
pad  for  command  marker  setting 

ALTITUDE 

“Maintain  altitude  XXX” 

Desired  altitude  displayed  in  the  scratchpad 
for  command  marker  setting 

HEADING 

“Come  Right  (Left)  XXX 
degrees’* 

‘Turn  (right/Left)  Heading 
XXX’* 

Desired  heading  displayed  in  the 
scratchpad  for  setting  of  the  heading  “bug” 

COURSE 

‘Turn  (Right/Left)  Heading 
XXX** 

“Maintain  Course  XXX’* 

Desired  course  displayed  in  the  scratchpad 
for  setting  of  the  course  on  the  HSI 

Table  1.  Deviation  Prompts  of  the  APES  “Advisory  Copilot”  Component 
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PROCEDURAL  PROMPTS 


DECISION  AID 
PARAMETER 

AUDITORY 

VISUAL 

TURN 

‘Turn  (Direction)  Heading 
(XXX)*‘ 

Textual  display  of  direction  appeared  on 
the  right  CRT  and  in  the  scratchpad. 

AIRSPEED 

“Set  Airspeed  XXX  knots** 
“Maintain  Airspeed  XXX  knots’* 

Displayed  desired  airspeed  on  the  right 
CRT  and  in  the  scratchpad.  Reversed 
video  highlight  Airspeed  Select  Button  on 
lower  CRT  to  cue  for  quick  entry  of 
command  value. 

ALTITUDE 

“XXX  feet  (above/below)” 
“Minimum  Altitude  XXX  feet” 
“Begin  { Descent/Climb)” 

Displayed  desired  altitude  on  right  CRT 
and  in  scratchpad.  Reversed  video 

highlight  Altitude  Select  Button  on  lower 
CRT  to  cue  for  quick  entry  of  command 
value. 

NAVAID 

“Set  Navaid  Channel  and  /  or 
Frequency  (XXX  /  XXX.XX)” 

Displayed  desired  freq./ch  on  right  CRT 
and  in  scratchpad.  Reversed  video 

highlight  Navaid  Select  Button  on  lower 
CRT  to  cue  for  quick  entry  of  Navaid 
Channel  and/or  Frequency. 

COURSE 

“Set  Inbound/Outbound  Course 
(XXX)” 

Displayed  desired  course  on  right  CRT 
and  in  scratchpad.  Reversed  video 
highlight  Course  Select  Button  on  lower 
CRT  to  cue  HSI  course  entry. 

Table  2.  Procedural  Prompts  of  the  APES  “Advisory  Pilot”  Component 


intercept  that  course. 

Table  2  shows  the  various  types  of  procedural  advice  of 
the  APES  “advisory  pilot”  component  The  content  of 
procedural  advice  differs  with  the  phase  of  the  approach 
(i.e.,  holding,  initial  approach,  final  approach,  missed 
approach).  For  example,  when  it  is  time  to  turn  the 
aircraft  onto  the  holding  inbound  course,  the  APES 
“advisory  pilot”  component  gives  verbal  prompts  and 
notifies  the  pilot  of  the  appropriate  heading  and  direction 
of  turn  to  fly  onto  that  course. 

APES  voice  messages  are  reinforced  with  textual  output 
which  are  displayed  on  the  right  CRT  in  the  MAGIC 
simulator  (see  Figure  3).  The  APES  continually 
displays  updated  target  values  for  radio  channel/ 
frequency,  altitude,  airspeed,  heading,  and  course  in  a 
scratchpad  area,  located  on  MAGIC’s  center  CRT  to 
allow  appropriate  command  and  course  settings.  APES 
fficn  displays  command  settings  for  radio,  altitude, 
airspeed,  heading  and  course  on  a  dedicated  CRT. 

5.  PILOT-IN-THE-LOOP  EVALUATION 

Upon  the  completion  of  the  APES  prototype,  a  pilot-in- 
the>loop  evaluation  was  conducted  to  assess: 

1)  the  effectiveness  of  APES  for  supporting  approach 
tasks  and  its  potential  for  reducing  pilot  workload, 
increasing  situational  awareness,  and  improving 
performance. 

2)  the  performance  of  the  decision  aid  to  determine  if 
APES  advice  was  accurate  and  timely  enough  to 
assist  the  pilot  in  flying  instrument  approaches. 

3)  the  understandability  and  useability  of  the  pilot- 
vehicle  interface. 

To  accomplish  these  objectives,  16  pilots  flew  a  series  of 
instrument  approaches  in  the  MAGIC  simulator.  The 


presence  or  absence  of  the  decision  aid,  the  orientation 
of  the  electronic  approach  plate  (North-Up/Track-Up), 
and  task  difficulty  (high/low)  were  varied  across  testing 
sessions.  In  the  high  task  loading  condition,  pilots  flew 
non-precision  approaches,  with  wind  gusts  incorporated 
into  the  aeromodel,  and  were  given  no  prior  review  of 
the  approach.  In  the  low  task  loading  condition,  pilots 
flew  precision  approaches,  without  wind  gusts,  and 
reviewed  the  approach  plate  prior  to  flying  the  approach. 
Right  performance,  workload,  situational  awareness,  and 
questionnaire  data  were  collected  and  analyzed. 

The  results  showed  that  the  APES  enhanced  the  pilot’s 
ability  to  perform  instrument  approach  tasks  compared  to 
flying  the  approaches  without  APES  assistance.  With 
apes  assistance,  pilots  deviated  significantly  less  from 
assigned  altitudes  during  high  task  loading.  See  Figure 
5. 


Rgure  5.  Task  Loading  as  a  Function  of  Altitude 
Deviation 

They  also  deviated  less  from  assigned  airspeed  during 
the  initial  and  final  approach  phases.  Using  the 
Subjective  Workload  Dominance  (SWORD)  technique, 
pilots  rated  their  workload  lower  with  APES,  especially 
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during  high  task  loading  conditions  (see  Figure  6),  and 
their  situational  awareness  higher  with  APES, 
independent  of  task  loading.  Study  findings  also  indicate 
that  APES  effectiveness  was  not  influenced  by  the 
orientation  of  the  electronic  approach  plate  (i.e.,  north- 
up  or  track-up). 

Regarding  the  performance  of  the  decision  aid  itself, 
pilots  rated  APES’  accuracy,  consistency  and  timeliness 
as  above  "moderately  acceptable,"  however,  pilot 
comments  indicated  that  the  deviation  tolerance  windows 
for  APES  voice  activation  may  have  been  too  stringent, 
especially  the  airspeed  and  altitude  voice  prompts.  Also, 
the  timeliness  of  the  procedural  prompts,  given  at  the 
approach  fixes,  were  reported  as  being  slightly  "hurried." 
Pilots  rated  the  APK  PVI  (audio  and  visual)  as 
"acceptable,"  although  some  refinements  were  indicated. 

Regarding  its  overall  utility,  two-thirds  of  the  pilots  rated 
the  overall  utility  of  the  decision  aid  for  assisting  in 
instrument  approaches  as  “extremely  beneficial.’’  Pilots 
thought  the  decision  aid  enhanced  performance  on  all  of 
the  approach  tasks.  Most  pilots  commented  that  the 
decision  aid  should  improve  flight  safety;  however, 
several  pilots  expressed  concern  with  the  consequences 
of  being  over-reliant  on  the  system. 


6.  GENERAL  ISSUES  TO  CONSIDER/ 
IMPLICATIONS  FOR  FURTHER  RESEARCH 

As  with  any  decision  aid,  careful  consideration  needs  to 
be  given  to  the  known  disadvantages  associated  with 
semi-automated  systems.  Because  low-level  decision 
making  processes  are  automated,  the  pilot  may  view  the 
system  as  a  black  box  that  generates  outputs  from  inputs 
through  some  unknown  mechanism,  such  as  the 
algorithm.  This  may  impair  pilot  confidence  in  the 
system  and  result  in  the  pilot  completely  ignoring  the 
advice  of  the  decision  aid.  Conversely,  too  much  trust 
and  over-reliance  on  the  decision  aid  may  lead  to 
reduced  pilot  situational  awareness,  which  in  turn,  could 
adversely  affect  flight  performance  if  the  decision  aiding 
system  fails  or  an  emergency  occurs.  Proper  training  of 
the  decision  aid  logic  is  imperative  to  the  success  of  a 
decision  aid,  enabling  the  pilot  to  develop  an  accurate 
mental  model  of  the  reasoning  behind  the  advice. 
Equally  important  is  proper  design  of  the  pilot-vehicle 


interface  that  will  allow  the  pilot  to  easily  interpret  the 
meaning  of  the  decision  aid  advice. 

The  results  of  this  study  strongly  encourage  future 
development  of  the  Approach  Proc^ure  Expert  System 
(APES).  Enhancement  of  the  APES  algorithm  is  one  area 
for  follow-on  research.  Future  implementations  of 
APES  altitude  and  airspeed  deviation  advice  may  need  to 
consider  trend  information.  In  its  current 
implementation,  APES  could  not  provide  explicit 
corrective  altitude  and  airspeed  advice  (i.e., 
increase/decrease  by  XXX  feet  or  XX  knots)  because 
altitude  and  airspe^  were  too  dynamic  for  accurate 
input.  APES  pnx^ural  advice  could  also  be  augmented 
to  provide  a  predictive  rate  of  descent  (i.e.,  WI)  to 
capture- target  altitude. — In  its  ^urrent-titylementalion,- 
APES  only  periodically  prompted  the  pilot  of  the  target 
altitude. 

Future  pilot-in-the-loop  studies  should  evaluate  APES  to 
assess  its  robustness  across  a  diversity  of  approach  types, 
including  high-altitude  approaches  (only  low-altitude 
were  used  in  the  current  study)  and  non-typical 
approaches.  Future  APES  implementations  will  also 
ne^  to  address  potential  conflicts  between  the  APES 
audio  advice  and  radio  communications.  This  research 
should  investigate  pilot-selectable  options  for 
configuring  the  PVI  (e.g.,  setting  deviation  tolerance 
windows  for  decision  aid  activation).  Issues  involved  in 
implementing  an  intelligent  agent  for  automatic 
adjustments  of  display  and  advice  settings  should  also  be 
examined.  Finally,  avenues  should  be  investigated  to 
determine  the  benefits  of  applying  APES  to  other  phases 
of  flight,  such  as  departures,  airdrop  and  air-refueling 
operations  and  low-level  routes. 
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SUMMARY 

Drawing  on  explanative  models  of  team  behaviour  and  cognition,  this  paper  seeks  to  describe  the  challenges  faced 
by  ad-hoc  and  distributed  military  teams,  which  are  increasingly  common  structures  in  modern  warfare  seUings. 
These  comments  are  made  in  the  context  of  supporting  evidence  drawn  from  a  recent  survey,  targeted  at  individuals 
with  experience  of  operating  in  these  teams.  Issues  addressed  include  the  maintenance  of  essential  teamwork 
behaviours,  the  development  of  shared  situation  awareness,  and  the  special  challenges  faced  by  team  commanders. 
Distributed  and  ad-hoc  teams  are  examined  with  the  added  aspect  of  one,  or  more,  members  of  the  team  potentially 
being  electronic  crewmembers  (ECs).  It  is  concluded  that  for  these  team  settings,  developers  of  future  ECs  should 
consider  moving  away  from  a  functional  model  of  Task  manager’  for  the  EC,  to  one  of  team  process  facilitator  . 


INTRODUCTION 

Distributed  teams  can  be  either  a  team  with 
distributed  skills,  or  more  commonly  one  whose 
members  are  in  some  way  physically  dispersed,  but 
who  are  operating  towards  the  achievement  of  shared 
goals.  Ad-hoc  teams,  consist  of  individuals  brought 
together  temporarily  for  the  achievement  of  short¬ 
term  goals,  who  are  likely  to  have  little,  or  no 
previous  experience  of  working  together.  The  use  of 
ad-hoc  teams  in  distributed  structures  is  now  typical 
of  many  modem  Operations  Other  Than  War 
(OOTW)  and  Joint  Force  operations. 

In  recent  years,  increasing  research  has  been  devoted 
to  the  application  of  Artificial  Intelligence  (AJ)  and 
Knowledge  Based  Systems  (KBS)  to  aircraft  cockpit 
environments.  This  has  been  reflected  in  the 
development  of  electronic  crewmembers  (ECs)  to 
support  pilot  task  conduct.  In  broad  terms,  much  of 
the  existing  EC  functionality  is  focused  on 
monitoring  system  status,  intelligently  managing 
displays,  dealing  with  system  malfunctions,  and 
offering  situation  assessment  and  mission  planning 
advice.  It  is  likely  that  further  emphasis  will  be 
placed  on  dynamic  function  allocation  (DFA)  and 
increasing  levels  of  system  decision  autonomy  within 
future  EC  prototypes  In  addition,  thought  has  been 
given  to  expanding  the  EC  concept  to  larger 
teams/crews  (e.g.  [1]).  Inevitably,  this  also  means 
assessing  the  opportunities  for  exploiting  the  EC 
concept  in  teams  situated  in  the  land  systems  and 
naval  domains.  With  this  in  mind,  and  with  the 
military’s  trend  to  use  both  distributed  and  ad-hoc 


teams  on  a  larger  scale,  it  is  important  to  ask 
questions  about  the  impact  and  consequences  of 
using  EC’s  in  these  team  structures,  particularly  those 
outside  of  the  aircraft  cockpit. 

Research  currently  being  conducted  at  DERA  Fort 
Halstead  (e.g.  [2])  is  seeking  to  better  understand  the 
unique  challenges  faced  by  ad-hoc  and  distributed 
military  teams,  and  to  develop  support  techniques  to 
improve  team  effectiveness  in  these  settings. 
Drawing  on  explanative  models  of  team  behaviour 
and  cognition,  the  remainder  of  this  paper  will  outline 
some  of  the  difficulties  faced  by  teams  in  these 
settings.  Evidence  is  also  presented  from  a  recent 
survey,  targeted  at  individuals  with  experience  of 
operating  in  these  team  structures.  In  doing  so,  the 
implications  for  the  insertion  of  one,  or  more,  ECs 
into  ad-hoc  and  distributed  teams  will  be  considered. 


TEAM  INTERACTION  CHALLENGES 

The  research  at  DERA  Fort  Halstead  has  examined 
ad-hoc  and  distributed  team  performance  from  a 
number  of  behavioural  model  perspectives.  Morgans’ 
et  al  (94)  [3],  theoretical  model  of  team  development 
provided  a  useful  starting  point  for  this  work.  They 
argue  that  there  are  two  distinguishable  tracks  that 
develop  within  the  team  as  it  matures.  There  is  the 
taskwork  track,  which  centres  on  the  operations- 
related  activities  undertaken  by  the  team.  The  second 
track,  teamwork,  is  represented  in  those  behaviours 
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and  activities  that  enhance  the  interrelationships, 
communication,  co-operation  and  cohesion  within  the 
team.  It  is  argued  that  through  a  series  of 
development  stages  (i.e.  forming,  storming, 
norming),  these  two  tracks  gradually  coalesce  to 
facilitate  effective  and  co-ordinated  performance 
within  the  team.  The  importance  of  this  model,  is 
that  this  paper  will  argue  that  the  traditional 
functional  model  for  the  EC  may  not  be  appropriate 
for  effectively  supporting  those  activities  associated 
with  the  teamwork  track,  particularly  in  ad-hoc  and 
distributed  settings. 

Building  on  the  team  evolution  model,  McIntyre  and 
Salas  (95)  [4]  identified  a  series  of  essential 
teamwork  behaviours,  which  underpinned  effective 
team  performance.  These  were  embodied  in  twenty 
principles.  Key  amongst  these  behaviours  is 
performance  monitoring,  feedback,  closed-loop 
communication  and  backing-up  behaviours. 
Researchers  at  DERA  Fort  Halstead  used  these 
principles  as  a  basis  for  a  survey  of  the  teamwork 
experiences  of  twenty-nine  individuals  operating  in 
military  ad-hoc  and  distributed  settings.  Based  on 
this  research,  the  following  observations  are  made 
about  these  teamwork  settings  and  the  implications 
for  EC  development,  beginning  with  the  four 
essential  teamwork  behaviours  described  above;- 

Monitoring;-  In  order  to  maximise  team  efficiency 
and  to  build  up  a  psychological  contract  of  trust 
among  team  members,  the  monitoring  of  fellow 
members’  performance  is  considered  critical  [4]. 

Monitoring  in  distributed  teams  has  to  be  largely 
conducted  using  various  electronic  links. 
Unfortunately  this  results  in  a  loss  of  the  richness  of 
information  passed  between  members,  making 
accurate  situation  assessment  of  team  and  system 
states  difficult  [5].  In  the  survey  conducted  by 
DERA  researchers,  86%  of  respondents  argued  that 
monitoring  fellow  team  member  performance  was 
more  difficult  than  in  a  co-located  setting.  A  typical 
comment  was  that  it  was  ‘easier  for  people  to  hide 
their  anxieties  and  problems’. 

With  ad-hoc  teams,  individuals  may  be  reluctant  to 
monitor,  or  may  be  unaware  of  the  need  to  monitor, 
others  performance.  Ad-hoc  teams  have  the 
additional  problem  that  they  typically  have  limited 
opportunities  to  train  together  and  thus  build  up  team 
norms  associated  with  the  establishment  of  team 
identity.  Consequently,  knowledge  of  fellow 
members’  strengths  and  weaknesses  is  limited,  and 
team  interdependence  is  weakened. 


Monitoring  -  EC  implications;-  Often  interaction 
with  the  EC  has  to  follow  strict  protocols,  resulting  in 
a  reduction  of  available  cues  to  monitor.  In 
particular,  the  EC  currently  has  little  prospect  of 
being  able  to  interpret  critical  cues  associated  with 
anxiety,  workload,  fatigue,  and  risk-taking  in  team 
members.  In  addition,  team  settings  are  dynamic, 
with  the  team  altering  its  task,  function  allocation  and 
workload  structure  accordingly.  This  begs  the 
question  of  what  the  EC  should  monitor,  when,  and 
how  it  could  be  made  responsive  to  changing  team 
circumstances.  It  has,  however,  been  suggested  that 
the  EC  could  monitor  task  progress  and  completion, 
and  make  that  information  available  to  other  team 
members.  Alternatively,  it  could  prompt  the 
Commander,  or  other  individuals,  to  actively  monitor 
team  members’  progress. 

Feedback;-  As  a  result  of  monitoring  team 
performance,  information  on  team/task  performance 
should  be  made  available  to  other  team  members. 
This  is  a  critical  process  for  building  up  adequate 
levels  of  shared  situation  awareness  within  the  team. 
More  generally,  feedback  also  facilitates  team 
maturation,  enabling  individuals  to  develop  an 
understanding  of  team  norms  for  appropriate 
intervention. 

As  with  the  process  of  monitoring,  it  is  proposed  that 
an  ad-hoc  team  with  an  immature  sense  of  team 
identity  and  limited  appreciation  of  team  norms,  will 
find  it  extremely  difficult  to  provide  and  accept 
constructive  feedback.  This  was  confirmed  by  62% 
of  survey  respondents.  In  addition,  nearly  80%  of 
respondents  believed  that  the  same  was  true  of 
distributed  environments,  in  comparison  with  co¬ 
located  settings.  Once  again,  it  is  postulated  that  the 
reason  for  this  finding  lies  in  the  degraded  quantity 
and  quality  of  cues  available  across  electronic  media, 
from  which  the  individual  has  to  try  and  develop 
appropriate  responses. 

Feedback  -  EC  implications;-  It  has  been  suggested 
that  one  of  the  biggest  obstacles  to  human 
recognition  of  expert  systems  errors  is  the  poor 
quality  of  feedback  about  the  activities  of  the 
automated  systems,  such  as  EC’s  [6].  Therefore,  if 
other  members  of  the  team  are  to  effectively  exploit 
the  EC,  it  is  imperative  that  the  feedback  provided  by 
the  system  on  its  performance  matches  human  team 
members’  expectations.  This  could  be  in  terms  of 
content,  time-scales  for  delivery  of  the  feedback,  and 
perhaps  most  importantly,  appropriateness  of 
feedback  given.  This  might  vary  as  a  function  of  the 
experience  of  the  other  team  members,  and  the  range 
of  operational  situations  encountered.  The  problem 
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of  team  turnover,  resulting  in  ad-hoc  team  structures, 
is  likely  to  add  to  what  is  a  taxing  challenge  for  EC 
developers. 

However,  if  the  EC  was  positioned  less  as  an  integral 
decision  making  member  of  a  team,  and  more  as  an 
overseer  of  team  process,  it  is  possible  to  envisage 
how  it  could  be  used  to  prompt  the  initiation  of 
feedback  activities  within  the  team.  This  could  prove 
particularly  useful  in  those  ad-hoc  team  settings 
where  interdependence  is  weak,  and  there  may  be 
reluctance  to  engage  in  these  essential  behaviours. 

Closed-loop  communication;-  Although  effective 
communication  in  any  team  setting  is  important,  the 
process  of  ‘closed-loop’  communication  is 
considered  to  be  essential  [4].  This  is  where  an 
individual  acknowledges  receipt  of  information  and 
the  sender  checks  that  the  message  has  been 
interpreted  correctly. 

These  communication  protocols  may  be  common  to 
military  settings,  however  it  is  important  to  note  that 
shared  communication  does  not  automatically  result 
in  shared  understanding  [7].  In  essence,  without 
clarification  or  cross-checking,  it  is  possible  that 
messages  may  not  be  interpreted  correctly  by 
recipients.  A  typical  comment  drawn  from  the 
DERA  survey  was  that,  ‘communications  are  the 
single  biggest  problem  in  all  distributed  operations’. 
Naturally,  the  number  and  nature  of  communication 
links  in  distributed  settings  is  somewhat  limited,  with 
65%  of  DERA  survey  respondents  confirming  that 
communicating  over  electronic  media  is  not  as 
effective  as  face-to-face  interaction.  The 
consequence  is  that  fewer  cues  exist  from  which  the 
sender  of  the  information  can  draw  accurate 
inferences  concerning  recipient  interpretation  of  that 
data. 

Unfamiliarity  with  fellow  team  members  in  ad-hoc 
teams  also  contributes  to  the  difficulty  of  interpreting 
the  meaning  of  restricted  response  information.  In 
addition,  it  is  also  clear  that  the  checking  mechanisms 
required  in  effective  closed-loop  communication  are 
less  likely  to  be  conducted  in  ad-hoc  teams.  It  is 
hypothesised  that  this  may  be  a  function  of 
inconsistent  mental  models  held  by  team  members, 
leading  to  false  assumptions  concerning  recipient 
understanding  of  data.  This  issue  is  discussed  in 
more  detail  later. 

Closed-loop  communications  -  EC  implications;- 
A  study  of  collaborative  work  on  intellectually 
demanding  tasks  [8],  highlighted  the  criticality  of 
informal  (i.e.  short,  unplanned,  unstructured) 
communications.  Like  other  computer  mediated 


communication  systems,  current  ECs  largely  utilise 
‘pre-planned,  relatively  structured  interactions  with 
pre-selected  communication  partners’.  In  short,  the 
opportunity  to  exploit  informal  interaction  to  enhance 
understanding  and  interpretation  of  information  with 
ECs  is  restricted-  A  significant  challenge  for  the 
developers  of  future  ECs  is  to  incorporate 
mechanisms  for  checking  or  ensuring  that  recipients 
understand  information  correctly.  For  example,  this 
could  be  reflected  in  appropriate  message  prompts 
that  help  confirm  shared  understanding  of 
commander’s  intent  and  team  goals/roles.  This 
support  should  also  seek  to  resonate  evolving  state 
changes  in  this  information. 

Backing-up  behaviours;-  The  fourth  essential 
teamwork  principle,  is  embodied  in  the  teams 
willingness  and  preparedness  to  back  fellow 
members  up  during  operations. 

Nearly  80%  of  those  questioned  in  the  DERA  survey 
believed  that  it  was  more  difficult  for  members  of  ad- 
hoc  teams  to  know  how  and  when  to  support  (i.e. 
provide  assistance  to)  each  other,  compared  with 
mature  teams.  These  concerns  can  be  summarised  in 
the  following  survey  comment;  ‘there  are  natural 
inhibitions  between  people  who  are  unfamiliar  with 
each  other.  This  is  exacerbated  where  the  ad-hoc 
team  is  joint,  combined,  or  coalition  in 
nature... where  the  more  subtle  uses  of  language  are 
unfamiliar  to  one  party  and  may  be  lost  to  other  team 
members’.  Ad-hoc  teams  do  not  have  well- 

developed  norms,  or  levels  of  trust,  that  facilitate  the 
practice  of  backing-up  behaviours.  Individuals  do 
not  know  the  strengths  and  weaknesses  of  their 
colleagues  and,  as  was  also  observed,  ‘are  naturally 
preoccupied  with  orienting  themselves  to  the  new 
task/team  environment’.  In  a  different  way,  these 
problems  are  also  common  to  distributed  team 
settings.  Even  if  team  members  have  a  greater  sense 
of  interdependence,  restricted  modes  of 
communication  degrade  important  cues  concerning 
stress,  fatigue  and  workload.  Thus,  it  is  not 
reluctance,  but  a  lack  of  pertinent  information  upon 
which  to  base  judgements,  that  delays  timely 
interventions. 

Backing-up  behaviours  -  EC  implications;-  If 

team  members’  are  to  accept  back-up  from  ECs,  then 
they  must  have  implicit  trust  in  the  information  or 
assistance  provided  by  the  system.  Hicks  and  Ross 
(1990)  [9]  addressed  the  issues  of  trust  and 
acceptance  in  the  cockpit,  of  human/electronic 
teamwork.  They  found  that  overall,  pilots  welcomed 
automation  that  would  relieve  them  of  tasks  during 
periods  of  high  critical  workload,  and  managed 
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mundane  and  routine  monitoring  tasks.  However, 
they  also  noted  that  there  was  a  great  deal  of  mistrust 
and  scepticism  concerning  the  integrity  and  reliability 
of  the  automated  systems.  Thus,  if  an  automated 
system,  such  as  an  EC,  is  flawed  in  some  way, 
causing  an  error,  trust  in  that  system  is  easily 
destroyed.  In  human  teams,  such  breaches  of  trust 
can  be  repaired,  on  the  basis  of  confidence  in  humans 
learning  from  mistakes  made.  The  recovery  of 
confidence  in  the  EC  (without  the  explicit  exposure 
of  a  similar  learning  process)  is  arguably  far  less  easy 
to  achieve. 

As  with  the  other  teamwork  behaviours  discussed,  a 
potential  EC  will  always  encounter  difficulties  in 
knowing  when  to  intervene  and  engage  in  backing-up 
behaviour.  The  current  paucity  of  automated  systems 
in  the  land  systems  domain  means  that  there  is  an 
absence  of  input  information  upon  which  a  potential 
EC  could  operate  decision  rules  governing  such 
interventions.  Should  such  infrastructures  be 
developed,  then  an  EC  overseeing  the  teamwork, 
could  monitor  the  progress  of  tasks  against  stated 
deadlines,  and  suggest  to  the  team  leader  when 
potential  problems  are  arising.  For  example,  this 
could  take  the  form  of  advice  concerning  the  re¬ 
distribution  of  workload,  and  might  be  particularly 
useful  in  those  team  structures  where  the  team  leader 
does  not  have  direct  sight  of  their  team.  It  could  also 
prove  to  be  useful  in  ad-hoc  teams  whose  members 
do  not  have  the  operational  experience  to  know  when 
or  how  to  engage  in  backing-up  behaviours.  In  the 
short  term,  a  simple  prompt  function  reminding  team 
leaders  to  check  team  member  status,  could 
potentially  promote  timely  backing-up  behaviours 
within  the  team.  Similarly,  distributed  team  members 
could  also  be  prompted  for  feedback  on  current  and 
anticipated  workload  levels. 

Team  leadership;-  Seven  of  the  other  team 
principles  identified  by  McIntyre  and  Salas  [4]  focus 
on  the  critical  role  of  the  team  leader  in  ensuring  co¬ 
ordinated  team  and  task  performance.  These  include 
the  need  for  the  leader  to  foster  respect  with  the  team, 
to  act  as  a  role  model,  and  to  provide  and  receive 
feedback  within  the  team. 

As  one  respondent  to  the  DERA  survey  noted,  ‘the 
greatest  challenge  to  any  leader  is  creating  a  team 
from  a  group  of  individuals’.  Leaders  of  teams  must 
provide  a  model  of  teamwork,  which  promotes  leader 
perceptions  of  goals,  roles  and  responsibilities,  and 
expectancies  concerning  future  team  responses.  This 
model  can  be  difficult  to  communicate  to  ad-hoc 
team  members  who  may  bring  different  perceptions 
of  teamwork,  situational  understanding,  goals  and  in 


multi-national  teams,  even  conflicting  political 
agendas.  In  ad-hoc  and  distributed  settings,  it  is  more 
difficult  for  team  leaders  to  exercise  their  personal 
style  and  to  foster  respect  among  remote,  or  relatively 
unknown,  team  members’  [2].  It  is  also  more  difficult 
to  monitor  performance  and  to  offer  feedback  in 
distributed  settings.  Although  the  team  leader  has 
sight  of  team  products,  they  may  not  necessarily  see 
the  process  by  which  these  products  were  arrived  at. 

Team  leadership  -  EC  implications;-  The 

opportunity  for  an  EC  to  adopt  a  highly  autonomous 
team  leadership,  or  task  control  role  within  the  team 
settings  described  is  arguably  limited.  Commanders 
of  teams  have  to  make  decisions  in  an  information 
environment  characterised  by  high  levels  of 
uncertainty  and  ambiguity.  Situation  assessment  and 
decision  making  will  be  conducted  through  a  varying 
range  of  autocratic  or  consensual  team  processes. 
Developing  an  EC  that  reflects  this  adaptability  in 
team  leadership  strategies  would  be  a  challenging 
prospect.  Instead,  it  is  proposed  that  an  EC  could 
assist  a  team  leader  to  promote  an  effective  teamwork 
model  in  ad-hoc  and  distributed  teams.  It  could 
potentially  assist  in  the  rapid  promulgation  and 
confirmation  of  team  goals,  and  could  prompt,  or 
assist  the  team  leader,  to  monitor  and  offer  feedback 
within  the  team. 

Moving  away  from  the  largely  behavioural 
perspective  adopted  by  McIntyre  and  Salas  [4]  in 
examining  effective  team  performance,  the  remainder 
of  the  paper  will  examine  team  functioning  in  these 
settings  in  terms  of  team  cognition. 

Shared  Mental  Models:-  It  has  been  argued  that 
team  members’  exercise  shared  knowledge  bases  that 
enable  them  to  accurately  explain  their  task 
environment,  and  to  anticipate  the  requirements  and 
actions  of  other  team  members  [10].  These  shared 
mental  models  enable  a  team  to  adapt  successfully  to 
stressful  situations,  maintain  shared  situation 
awareness,  and  help  trigger  critical  team  behaviours, 
such  as  monitoring  and  feedback. 

Compared  with  mature  teams,  73%  of  DERA  survey 
respondents  believed  that  it  was  more  difficult  for  ad- 
hoc  teams  to  maintain  shared  situation  awareness.  In 
addition,  72%  of  respondents  also  asserted  that  ad- 
hoc  teams  were  not  as  effective  at  anticipating  the 
needs  of  their  team  members.  Similar  findings  were 
recorded  for  distributed  teams,  in  comparison  with 
co-located  group  structures. 

Ad-hoc  teams  lack  the  sense  of  identity  and 
interdependence  that  facilitates  the  formation  and 
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practice  of  shared  mental  models.  Limited  training 
time  together  as  a  team,  frequently  means  that  the 
development  of  adequate  plans,  contingencies  and 
anticipated  responses  to  possible  situations  may  not 
occur,  or  may  not  be  consistently  shared  across  the 
team.  In  addition,  team  norms  that  facilitate  the 
effective  use  of  essential  teamwork  behaviours  have 
often  yet  to  be  fully  developed.  The  leader  of  these 
teams  then  faces  the  daunting  task  of  trying  to 
quickly  establish  these  shared  views,  before  the  team 
must  swing  into  action.  This  can  be  particularly 
challenging  in  those  settings  where  the  team  must 
operate  in  a  distributed  structure. 

Shared  Mental  Models  -  EC  implications;-  The 

processing  of  automated  systems,  such  as  an  EC,  is 
not  easily  open  to  direct  review  (or  learning),  making 
it  difficult  for  team  members  to  generate  a  mental 
model  of  the  system  [7],  In  this  sense,  the  EC  is 
fundamentally  flawed  when  viewed  as  a  typical  team 
member  in  these  settings.  However,  it  is  suggested 
that  an  EC  could  potentially  prove  invaluable  in 
promoting  the  early  development  of  shared  mental 
models  and  in  assisting  the  maintenance  of  shared 
situation  awareness  in  changing  circumstances. 

Research  at  DERA  Fort  Halstead  has  recently  been 
initiated  to  establish  how  technology  can  be  exploited 
to  facilitate  the  exchange  of  information  on  goals, 
task  progress,  situational  understanding  and  resource 
management  within  a  team.  This  research  aims  to 
help  team  leaders  to  highlight  gaps  in  knowledge, 
resolve  conflicting  team  member  views  of  critical 
information  and  to  build  and  maintain  shared  mental 
models.  In  the  future,  a  sophisticated  EC  could 
therefore  help  build  up  consistent  mental  models  in 
an  ad-hoc  team,  and  maintain  shared  situation 
awareness  in  a  distributed  team.  The  dynamic 
prompting  of  essential  teamwork  behaviours  could 
also  enhance  this  process.  An  EC  might  also  help 
build  a  better  understanding  of  team  member 
strengths,  by  making  available  skill  profiles  in  a 
central  database,  and  by  providing  an  effective  means 
for  the  distribution  of  information  on  team  errors,  and 
the  recovery  of  those  errors. 


CONCLUSIONS 

In  summary,  it  can  be  argued  that  the  role  of 
decision-maker,  or  task  manager,  for  the  EC  in  larger 
teams,  particularly  those  outside  of  the  aircraft 
cockpit,  may  not  be  appropriate.  There  are  far  fewer 
electronic  systems  integral  to  the  planning,  situation 
assessment  and  decision  making  process  in  land 
operating  domains.  Thus,  there  are  currently  fewer 


interfacing  electronic  databases  that  can  be  exploited 
by  traditional  ECs.  More  importantly,  ad-hoc  and 
distributed  team  settings  are  dynamic,  with  the  team 
altering  its  task,  function  allocation  and  workload 
structure  accordingly.  In  addition,  decision  making 
and  situation  assessment  products  can  be  arrived  at 
through  a  varying  range  of  autocratic  or  consensual 
processes,  depending  on  a  range  of  impacting 
contextual  factors.  Developing  a  traditional  EC, 
which  is  this  adaptable  to  changing  team  decision 
strategies,  would  be  difficult.  The  ad-hoc  team  also 
lacks  the  stability  and  predictability  associated  with 
those  teams  who  exploit  well-practised  norms  as  part 
of  the  mature  team  process.  Consequently,  this 
makes  it  problematic  to  predict  in  advance  how  and 
when  a  potential  EC  should  support  team  taskwork 
activities. 

In  light  of  these  observations,  it  is  proposed  that  the 
potential  role  of  an  EC  in  these  team  settings  should 
be  as  a  ‘facilitator’  of  effective  team  process.  The 
EC  would  therefore  not  have  a  direct  controlling  role 
in  the  taskwork  activities  of  the  team.  Instead,  the 
EC  would  help  promote  effective  teamwork,  by 
prompting  the  conduct  of  essential  teamwork 
behaviours,  such  as  monitoring  and  feedback  within 
the  team.  The  EC  could  also  help  the  team  leader  to 
foster  an  early  sense  of  interdependency  and  identity 
within  the  team,  by  promoting  distribution  of  leader 
perceptions  of  team  goals,  member  roles  and 
interdependencies,  at  both  a  taskwork  and  teamwork 
level.  It  could  also  help  to  reflect  dynamic  changes 
of  command  intent  and  priorities  to  all  team 
members,  ensuring  continued  shared  situation 
understanding. 

It  is  also  argued  that  the  EC  could  have  a  key  role  to 
play  in  aiding  the  development  of  consistent  shared 
mental  models  within  the  team.  The  future  EC  could 
effectively  improve  metacognition  within  the  team, 
prompting  the  team  to  stand  back  from  the  decision 
making  and  situation  assessment  process,  and 
ensuring  that  knowledge  gaps  are  addressed,  that 
assumptions  are  consistent,  and  that  mental  model 
conflicts  are  swiftly  resolved. 

To  achieve  the  goal  of  an  effective  EC  team  process 
facilitator,  an  improved  understanding  of  the 
cognition  and  behaviour  underpinning  effective  team 
performance  needs  to  be  developed.  It  is  also 
recognised  that  there  are  many  technological 
challenges  associated  with  meeting  the  requirements 
of  the  electronic  team  facilitator.  However,  if 
designers  of  future  ECs  are  to  effectively  support 
team  process  in  larger  teams,  particularly  those  that 
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are  ad-hoc  or  distributed  in  structure,  then  these 
challenges  must  be  met. 
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Cognitive  Map  Displays 

SE  Finnie  and  RM  Taylor,  DERA  CHS,  Famborough  Hants,  UK 

This  paper  will  report  DERA.  CHS  work  on  cognitive  requirements  for  future  aircraft  digital 
map  di^lays  for  mission  situation  assessment,  planning  and  re-planning  tasks.  In  particular, 
the  paper  v^l  consider  the  human  factors  issues  of  applying  HEC  teamwork  concepts  to  the 
use  of  vector  data  map  displays,  supported  by  "intelligent"  decision  aiding  technologies.  It  will 
present  the  idea  of  a  Cognitive  Map  Display,  where  the  map  display  system  "knows  about" 
how  the  pilot  percieves,  thinks  and  acts.  It  will  demonstrate  the  implications  through 
protyping  of  a  cognitive  map  display  task. 

Despite  increasing  use  of  aircraft  Head-Up  Displays  (HUDs)  and  Helmet  Mounted  Displays 
(HMDs),  map  displays  continue  to  remain  the  most  complex  displays  of  information  in  the 
aircraft  coc^it.  Dynamic  mission  information  (position,  route,  mission  and  tactical 
information)  is  superimposed  on  complex  geographical  information.  The  term  Digital  Map 
Situation  Displays  (DMSDs)  has  been  coined  to  emphasise  the  important  interaction  between 
geographical  and  mission  information  that  occurs  at  the  display  surface.  Because  of  the  spatial 
nature  of  the  information,  and  of  die  nature  of  die  pilots  spatial  reasoning  map  tasks, 
positional  information  and  spatial  relationships  must  be  preserved.  Unlike  systems  and  flight 
information 

in  HUDs  and  HMDs,  the  ability  to  organise  (i.e.  separate,  integrate,  fuse,  chunk)  map  situation 
information  is  limited.  Spatial  position  is  not  a  DMSD  cognitive  compatibility  variable. 
Consequently,  information  superimposition  and  spacing  problems  often  produce  a  DMSD 
image  which  appears  cluttered,  which  is  difficult  to  search  and  to  interrogate,  and  often 
impossible  to  interpret  reliably  at  a  glance.  Extensive  design  effort  is  needed  on  DMSD 
information  coding,  symbology,  and  colour  to  even  approach  an  acceptable,  organised  image. 
Whilst  the  mission  tymboiogy  can  be  refined,  the  content  and  format  of  geogr^hical 
information  remains  largely  unchanged  from  hard  copy,  paper  maps.  Differences  may  occur  in 
the  information  presented  in  using  different  map  scales  for  the  required  look  ahead.  However, 
the  introduction  of  vector  mq>  data  bases  will  provide  tiie  capability  to  address  and  select 
individual  map  features. 

DMSDs  provide  information  for  the  support  for  the  critical  pilot  functions  and  decision 
making  in  mission  planning,  mission  management  and  control.  Support  for  in-fli^t  situation 
assessment  and  mission  re-planning  is  becoming  an  increasingly  important  issue.  EC  can 
provide  pilot  assistance  in  the  following  decision  making  tasks:  assessing  file  impact  of 
changes  in  the  tactical  and  threat  situation  on  the  mission  plan,  in  plan  repair  and  in  creating  a 
new  plan,  and  in  critiqueing  re-planning  with  reference  to  the  changing  mission  situation  and 
objectives.  Developments  in  digital  map  display  vector  data  bases  will  soon  provide  new 
flexibility  in  information  display  for  presentation,  organised  and  tailored  to  the  immediate 
situation.  With  EC  assistance,  this  flexibility  will  enable  the  support  to  be  more  closely 
matched  to  the  immediate  decision  making  needs  of  both  the  situation  and  the  user-  i.e. 
mission  and  pilot  context  sensitivity-  and  to  function  more  like  a  Cognitive  Map  Display. 
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EC  Support  for  FI  MotorSport 

Nikki  Heath,  Symbiotics  Ltd.,  UK 

The  Formula  One  industry  is  increasingly  aware  that  they  must  rely  on  race  tactics  and 
strategy  to  provide  consistent  performances.  Individual  championships  are  important,  but  the 
Constructors  Championship  carries  the  main  sponsorship  drive.  Therefore,  it  is  in  the  team's 
best  interest  to  coax  the  most  consistent  and  error  free  performance  out  of  their  drivers,  and 
provide  a  strategy  that  mirrors  the  driver's  strengths  and  weaknesses. 

The  Electronic  Crewmember's  ability  to  offer  the  most  direct  method  of  achieving  a  goal, 
would  enable  the  team  to  have  an  optimal  strategy  from  which  to  base  their  driver’s 
performance,  when  presented  with  a  particular  set  of  conditions.  Examples  would  include: 

.  Pit  Stop  Performance  such  as:  in-lap  speed,  pit  box  braking  point  and  angle  of  entry. 

.  Overtaking  strategy,  taking  into  account,  position  and  length  of  race,  track  conditions, 
weather,  car  ability  and  driver  ability. 

.  Risk  Taking.  Strategy  should  alter  according  to  the  individual  riskiness  of  a  driver.  It 
is  not  feasible  to  ask  a  driver  to  alter  his  fundamental  personality  to  fit  a  team  tactic  or 
car  set-up.  This  aspect  appears  to  pose  a  real  problem  in  some  teams. 

.  Number  of  pitstops  can  be  correctly  calculated  to  take  into  account  the  drivers  ability 
as  outlined  above.  In  addition,  factors  may  be  calculated  to  take  into  account  the  way  in 
which  the  driver  performs  with  a  full  fuel  tank  and  how  hard  he  wears  the  tyres  etc. 

.  Recording  of  behaviour  during  events  resulting  in  loss  of  control,  evaluation  of  the 
skills  that  were  eroded.  The  system  could  also  effectively  demonstrate  the  disparity  of  a 
driver's  actual  performance  in  terms  of  skill-based  responses  in  comparison  to  an  ideal 
set  of  responses. 

Mapping  and  predicting  an  individual's  behaviour,  would  enable  the  team  in  the  pits  to  alter 
car  set  up  and  tactics  as  the  events  of  a  race  unfold.  In  addition,  it  would  provide  a  very  useful 
tool  in  training  and  practice  for  improving  skills  and  knowledge  based  decision  making,  by 
offering  hard  evidence  of  sub-optimal  performances. 
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Combining  Crew  Assistant  Functions  of  Information  Management,  Data 
Link,  Mission  Planning  and  automated  Situation  Assessment  to  Improve 

Situation  Awareness 

by 

H.  Pongratz 

Daimler-Benz  Aerospace  AG 
Military  Aircraft  Division  (MLE) 

81366  Munich 
Germany 


1  SUMMARY 

The  crews  of  todays  fighter  aircraft  are  confronted 
with  workloads  that  often  exceed  tolerable  limits. 
The  demand  for  precision  attacks  in  the  presence  of 
multiple  constraints  and  the  necessity  to  avoid 
collateral  damage  further  complicate  the  tasks  of 
the  aircrews.  This  paper  describes  the  development 
of  crew  assistant  functions,  which  will  reduce  the 
crew  workload  and  improve  situation  awareness  of 
aircrews  in  combat  situations.  The  core  of  these 
functions  assist  in  situation  assessment  and  situati¬ 
on  display  to  the  crew  on  air  attack  combat  aircraft. 
These  functions  are  supported  by  an  airborne  in¬ 
formation  management  system,  by  data  link  inter¬ 
netting  and  inflight  mission  planning  or  replanning 
functions,  that  have  to  be  integrated  and  used  to 
derive  the  actual  situation  with  regard  to  threats, 
targets  and  ownside  assets  contributing  to  mission 
success.  Further  the  required  man-machine  inter¬ 
face  for  presentation  of  the  processed  information 
to  the  crew  in  real  time  are  described.  These  functi¬ 
ons  will  first  be  tested  and  implemented  on  a  gro¬ 
und  based  simulator.  Later  a  flight  test  in  a  combat 
aircraft  is  envisioned. 


2  INTRODUCTION 

2. 1  Human  factors  and  situation 

Even  in  a  two  seat  air-to-ground  fighter  aircraft  the 
workload  of  the  aircrews  often  exceeds  the  limits, 
where  crews  can  deliver  optimum  performance. 
Flexibility  to  react  to  changing  situations  is  reduced. 
This  applies  especially  during  the  actual  combat 
phase  of  the  mission.  In  today’s  combat  aircraft 
informations  are  collected  by  a  number  of  separate 
sensors  like  Radar  Warning,  FLIR  and  Radar  and 
presented  on  separate  displays.  It  must  be  checked 
against  the  preplanned  mission  data,  leaving  the 
task  of  processing  and  fusing  ail  these  informations 
and  combining  it  with  the  mission  plan  and  data 
into  a  valid  situation  assessment  to  the  aircrew. 

This  creates  a  considerable  workload  for  the  crew, 
which  adds  on  top  of  the  workload  for  flying  the 
aircraft  and  operating  the  weapon  system.  Limitati¬ 
ons  in  display  representation  further  complicate  this 
task. 


Consequently  a  main  issue  ii\  present  programs  for 
onboard  crew  assistance  systems  is  the  enhance¬ 
ment  of  the  situational  awareness  of  the  crew  by 
implementing  automatic  functions  for  information 
and  data  fusion,  automatic  situation  assessment 
and  situation  display  to  the  crews. 


2.2  Integration  of  external  information  sources 
to  improve  situation  awareness 

Another  shortcoming  reducing  situation  awareness 
of  the  crew  is  an  information  gap  after  a  mission 
has  been  launched.  If  only  high  frequancy  voice 
radio  is  available  to  forward  last  minute  information 
on  target  and  threat  states  to  the  airplane,  little 
information  can  be  delivered  and  the  aircrews  have 
only  very  limited  onboard  capabilities  to  perform 
adjustments  in  the  mission  plan  and  execution,  if 
the  tactical  situation  has  changed.  Usually  target 
and  threat  information  is  several  hours  to  days  old 
and  a  significant  situation  change  results  in  a  missi¬ 
on  abort. 

If  the  mission  is  part  of  a  combined  air  operation, 
formation  keeping  and  coordination  with  support 
and  escort  aircraft  within  the  flight  must  be  pre¬ 
planned  and  performed  rather  rigidly,  leaving  little 
room  for  flexible  raction  to  changing  situations. 

These  shortcomings  can  be  reduced  significantly  by 
introducing  a  data  link  internetting  of  mission  air¬ 
craft,  command  authorities  and  intelligence  sour¬ 
ces,  which  can  be  maintained  throughout  the  missi¬ 
on. 


2.3  Integration  of  onboard  mission  planning 
and  execution  assistant  functions  to  maintain 
situation  awareness  in  changing  environments 

In  order  to  maintain  situation  awareness  in  a  chan¬ 
ging  environment  and  to  make  best  use  of  data  link 
and  other  information  and  the  onboard  real  time 
situation  assessment  function,  Inflight  mission 
planning  and  replanning  functions  and  mission 
execution  assistant  functions  are  added  to  the 
system. 

The  mission  planning  functions  allow  an  Inflight 
replanning  of  a  mission,  if  threat  or  target  situation 
changes  and  rebuild  a  situation  assessment  under 
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the  new  conditions  to  maintain  situation  awareness 
of  the  aircrew.  The  mission  planning  function  relies 
on  a  target,  threat  and  terrain  data  base  to  build  a 
situation  representation  within  the  mission  compu¬ 
ter  of  the  aircraft.  Terrain  masking  and  threat  capa¬ 
bilities  like  detection  ranges,  firing  ranges,  suscep¬ 
tibility  to  ownship  ECM  and  self  defence  measures 
must  be  modelled  Into  the  situation  assessment  as 
well  as  target  vulnerability  and  possible  ownship 
attack  maneuvers  to  create  optimum  attack  routes 
and  profiles  under  changing  conditions  and  to  as¬ 
sess  the  resulting  new  situation  correctly  and  dis¬ 
play  it  to  the  aircrew.  The  new  mission  parameters 
must  be  transferred  to  the  aircraft  navigation  and 
auto  attack  systems  after  approval  by  the  crew  and 
the  execution  must  be  assisted  to  keep  the  crew 
workload  at  acceptable  levels. 


3  DESCRIPTION  OF  THE  INFORMATION  MANA¬ 
GEMENT  AND  CREW  ASSISTANT  SYSTEM  FOR 
SITUATION  ASSESSMENT  AND  DISPLAY 

3, 1  Structure  of  the  Information  managemant 
and  crew  assistant  system 
The  information  management  and  crew  assistant 
system  consists  of  the  following  modules  (Rg.  1). 
The  crew  interface  handles  the  display  outputs  to 
the  crew  and  accepts  crew  commands.  The  main 
situation  display  is  one  of  the  weapon  system  of¬ 
ficer's  color  head  down  displays.  Due  to  the  com¬ 
plex  information  to  be  presented  a  color  coded 
display  must  be  used.  The  main  interface  is  with  the 
WSO.  The  information  managemnet  distributes  data 
from  and  to  the  crew  interface  to  the  functional 
modules.  These  are: 

-  situation  assessment  and  display 

-  threat  assessment 

-  mission  planning  and  execution 

-  data  link  processing 

-  data  bases  /  data  base  management 

The  data  link  module  is  connected  to  an  onboard 
MIDS-terminal,  which  can  be  connected  to  external 
MIDS  or  JTIDS  ground  or  airborne  terminals  within  a 
JTIDS  net. 


3.2  Functional  modules  of  the  information 
management  and  crew  assistant  system 

3.2.1  Situation  assessment  module 

The  situation  assessment  module  performs  eight 

main  tasks: 

-  determine  all  known  threats  along  the 
attack  route 

-  compute  threat  ranges,  terrain  masking 
and  resulting  threat  areas 

-  compute  threat  cost  functions 


-  determine  threat  total  cost  along  pre¬ 
planned  flight  path  and  assess  situation 

-  compute  situation  display  formats 

“  monitor  threats  and  invoke  threat  eva¬ 
sion,  if  threat  gets  in  firing  range 

-  monitor  fuel  reserves  and  own  assets 
and  invoke  replanning,  if  reserves  are 
low 

-  monitor  timing  and  time  over  target 
and  adjust  command  speed 

-  update  threat  arjd  target  data  base 
from  ownship  sensor  ond  data  link  data 

If  a  route  planning  based  on  the  situation  asses- 
ment  results  in  too  high  risk  or  too  high  fuel  con¬ 
sumption  or  too  late  time  over  target,  the  route  will 
be  recomputed  with  ECM  applied,  which  usually 
should  result  in  a  shorter  route  and  /  or  reduced 
threat  risk. 

If  this  still  is  not  sufficient,  at  pilot's  request  the 
allowable  risk  factor  can  be  elevated  and  a  more 
direct  route  with  higher  risk  can  be  seledted  or  the 
mission  can  be  aborted. 

The  route  planner  also  can  be  forced  to  circumnavi¬ 
gate  "no  fly"  areas  and  fly  through  predetermined 
corridors  for  ingress  and  egress  from  the  target 
area. 


3.2.2  Threat  evasion  module 

The  threat  evasion  module  is  (Fig.  3)  invoked  ty  the 
situation  assessment  module,  if  a  known  threat 
comes  into  firing  range.  Main  tasks  of  the  threat 
evasion  module  are: 

-  compute  radar  burnthrough  range 
against  ownship,  request  ECM,  if  appli¬ 
cable  and  sufficient  to  neutralise  the 
threat 

-  run  onboard  SAM  simulation,  compu¬ 
ting  possible  missile  shots  at  the  owns¬ 
hip  to  determine  the  potentially  most 
threatening  shots  from  any  SAM  station 
in  range  and  alert  the  crew  and  propo¬ 
se  the  best  available  defensive  action 

-  determine  timing  and  flight  path  of  op¬ 
timum  evasive  maneuver  with  or 
without  ECM  support  (chaff  or  towed 
decoy) 

-  determine  timing  and  flight  director  in¬ 
dicator  for  evasive  maneuver,  adjust 
commanded  speed 

-  compute  situation  display  format  for 
self  defence  situation 

3.2.3  Route-  and  attackolanner  module  of  mission 
planning 

The  route-  and  attackplanner  module  (Fig.  4)  is 
invoked  by  the  situation  assessment  module  after  a 
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deviation  from  the  preplanned  course  or  after  chan¬ 
ges  in  the  threat  or  target  situation  or  if  own  assets 
run  low  and  prohibit  execution  of  the  preplanned 
mission  or  by  crew  request.  Main  tasks  of  the  route- 
and  attackplanner  module  are: 

-  compute  threat  optimized  route  from 
ingress  point  to  the  attack  corridor  and 
back  to  the  egress  point 

-  support  radar  navigation  updates 

-  support  precision  target  aimpoint  loca¬ 
tion 

-  compute  flyable  route  from  optimized 
route  with  low  level  profile 

-  perform  formation  planning  for  multis¬ 
hip  formation  (up  to  four),  deconfliction 
in  target  area  and  coordination  with 
mission  control  via  data  link 

-  compute  waypoints,  command  altitude 
and  speed  and  attack  computer  set¬ 
tings 

-  compute  fuel  consumption 
compute  timing  and  time  over  target 

-  compute  planning  display  format 

Main  purpose  of  these  functions  is  to  permit  Inflight 
retargeting  and  replanning  of  a  running  mission  and 
to  react  in  real  time  on  external  situation  changes. 

Another  application  is  for  missions  against  mobile 
targets,  which  can  be  set  up  completely  only  after 
the  flight  is  In  the  air  and  using  a  reconnaissance- 
attack  interface  via  data  link  to  direct  the  attack 
force  to  the  target  in  real  time. 


3.2.4  Situation  display  module 
The  situation  display  module  collects  information 
from  the  situation  assessment,  threat  evasion  and 
mission  planning  modules,  decides  which  informati¬ 
on  is  most  urgent  for  the  crew  now  and  composes 
the  relevant  information  content  into  one  of  three 
main  situation  display  formats  (Fig,  5): 

-  situation  display 

-  threat  evasion  display 

-  pre-  /  replanning  display 


-  message  compostion  for  outgoing  in¬ 
formation 

-  operation  of  data  link  transmit  and 
receive  modules  within  a  selected 
JTIDS-network 


4  EVALUATION  OF  SYSTEM 

The  information  management  and  crew  assistant 
system  will  be  installed  on  a  two  seat  Tornado  flight 
simulator  and  can  be  tested  |n  a  simulated  air-to- 
ground  mission  scenario  with  simulated  opposing 
and  allied  forces.  There  the  merits  of  the  system 
can  be  evaluated  in  a  simulated  combat  environ¬ 
ment  and  necessary  improvements  to  the  system 
can  be  identified. 

In  a  second  campaign  this  evaluated  system  will  be 
installed  In  a  Tornado  test  aircraft  in  a  pod  based 
system  to  keep  the  necessary  aircraft  modifications 
as  small  as  possible.  This  system  will  be  flight  te¬ 
sted  with  an  AWACS-JTIDS  ground  station  as  data 
link  partner  station  and  evaluated. 


5  CONCLUSIONS 

The  presented  research  and  development  activities 
will  provide  an  advanced  cockpit  avionics  system 
capable  of  improving  the  aircrew's  situational  awa¬ 
reness  in  a  combat  situation,  reducing  the  crew 
workload  during  critical  situations  and  improving 
the  flexibility  of  aircrew  and  aircraft.  Furthermore 
the  interoperability  with  allied  air  forces  will  be 
improved  by  providing  better  command  and  infor¬ 
mation  flow  via  the  data  link  connection,  which  will 
be  especially  important  for  multinational  missions 
within  Rapid  Reaction  Forces. 


3.2.5  Data  link  processing  module 

The  data  link  processing  module  (Rg.  6)  is  running 
all  the  time  as  a  background  process  and  channels 
in-  and  outgoing  information  through  the  data  link. 
Main  tasks  are: 

-  unpacking,  decoding  and  selection  of 
incoming  data 

-  data  fusion  of  onboard  data  base,  on¬ 
board  sensor  and  incoming  data  link 
data 

-  extraction  of  relevant  command,  forma¬ 
tion  and  mission  planning  update  in¬ 
formation  from  incoming  data  stream 
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structure  of  Crew  Assistant  System 


Fig.  1 


Situation  Awareness  Module  for  Crew  Assistant 


inputs: 

o  Threatposition,  o  Planned 
'typerStatUS  Route, 
o  Fuel  consumpt. 
oOwnshlpPo^. 
Heading,Spoecl  R^fer.  Pt. 


Functions: 

o  Compute  ownship/ttireat 
Intervieibittty 

o  Compute  radar/SAM  threat 


o  Compute  terrairVWeather 
hazard  areas 

o  Update  Target/Threat  DB, 
Corporate  Trackftlee 

o  Derive  combined  coat 
function 

o  Situation  Aaaessment 

o  Start  OBS/Self  defence.  If 
threat  la  in  range 

o  Monitor  fuel  reservoa,  start 
replanning.  If  fuel  low 

o  Monitor  timing  and  Time 
over  Target ,  adjust  com¬ 
manded  speed 

o  Compute  Situation  Display 


Used  Data  Bases: 

o  Long/Lat/Altftude  Terrain  D8 
F=?AC  Areas,  RadarRefPoInts 

o  Ijow  Level  Rylng  Chart 

o  Threat  DB  (  radar  range.flro- 
Ing  range,  ECM  bumthrough 
range) 

o  Air  Space  Order  (Nofly) 

o  Support  (Jam  Corridor)  DB 

o  A/C-Data  (RCS,  Lift.  Drag. 
Fuel,  Thrust ) 

o  Weather  DB 


Outputs: 

o  Situation  /  Threat  Display 
o  Fuel ,  Expendabels  Status 
o  Time.  Formation  Situation, 
Commanded  Speed 
0  Threat  /  Terrain  Cost  Function 


Fig.  2 
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Threat  Evasion  Maneuver  Module  for  Crew  Assistant 


Inputs: 

o  ThreatposWon.  o  Piannad 
-type.'StatUS  Route 
o  OBS  request 
o  Ownshlp  Posit., 

Heading, Speed 


Functions: 

o  OBS(Onboard  SAM  Simu¬ 
lation)  and  Safetime  comp, 
of  engaged  SAM’s 

o  Compute  radar  bumthrou- 
gh  range,  reguest  ECM,  if 
applicable 

o  Compute  evasive  maneu¬ 
ver,  If  applicable  with  com¬ 
bined  chaff  or  decoy  appli¬ 
cation 

o  Compute  Right  Director 
for  evasive  maneuver, 
adjust  commanded  speed 

o  Compute  Situation  Display 
for  self  defence  situation 


Fig.  3 


Route*  and  Attackplannermodule  for  Crew  Assistant  Ground  Attack 


Inputs: 

0  Ingress  P. 

0  Target  Pos. 

0  Egress  PL 

0  Attack  Profile 

0  Nofly  Area 

0  Attack  Conid’s 

oThreattype 

0  Weapon  Rele¬ 

Position 

ase  Points 

oTerUP.RAC 

0  Time  ov.Target 

RadRefPts 

Functions: 

o  Comp,  throeitoptlmlxod  rou- 
to  from  tngroaspoint  to  tlie 
Attack  Conidor 

o  Support  Radar  Navigation 
Updatea 

o  Support  Precision  Target 
AImpoint  Location 


Used  Data  Bases: 

o  LongA..at/Altitude  TerralnOB 

o  Low  Level  Flying  Chart 

o  RRP-Hoad7Positlons,  FIAC 
Areas.  Initial/TenTiInal  Update 
Polnts.poss.  Attack  Corridors 

o  Air  Space  Order  (Nofly) 


"flyable*  Route  (Smoothed) 
with  tow  level  profile 


o  A/C-Data  (RCS,  Lift,  Drag, 
Fuel.  Thrust ) 


Threcrt  optimized  Attack- 
profile  and  Weapon 
Releasa  Point  computation 


o  Throat  Types  (  SAM  ranges. 
Radar  ranges  ) 


Pormatlonplannlng .  Decon- 
flletlan  (  Max.  A  A/C)  and 
Coordination  by  Lead  (DL) 

Command  altitude  profile. 
Waypoints  and  Attack 
Computer  Settings 

Compute  Command  Spe¬ 
ed  and  Fuel  Consumt^on 

Compute  Time  over  Target 


o  Attack  profiles,  weapon 
delivery  characteristics 


Outputs: 

o  Sel.  Attack 
Corr./Profile 
0  Waypoints 
0  fnitUP.TerUP 
o  Time  ov.Target 


o  Com.Alt.AGL 
o  Com.  Speed 
o  Sel,  RadRefRs 
(Rxpoints) 
o  Attack  Com¬ 
puter  Setting 


Situation  Display  presented  on  3  display  formats: 


Situation  Display 
(Digital  Map  +  Overlay) 

o  track-up,  detail 
current  position 
Overlay: 

o  Next  Waypoints 
o  Update  Points 
o  Targets 
o  Threat  Areas 
(Radar  shadow  zones, 
SAM  engagement  zo¬ 
nes  wfth/without  ECM) 
o  Formation  Keeping 
o  No  Ry/  Fragmentation 
hazard  zones 
o  MIDS-Incomming  mess- 
eiges  for  Data  Base  updt. 
o  RECCE  message  comp¬ 
osing  for  outgoing  MIPS 


Threat  Evasion  Display 
( Digital  Map  +  Overlay) 

o  track-up,  detail 
cunent  position 
Overlay: 

o  Threat  Area/Status 
o  Next  Waypoints 
o  Proposed  evasive 
Maneuver 

o  Active  Self  Defence 
Actions  (ECM.ChafO 
o  Useable  Jeim  Corridors 
puddy  SSJ/SOJ) 
o  Formation  keeping 


Pre/Replanning  Display 
( DIPLAS  +  Autorouter) 

o  north-up,  overview 
or  detail  target  area 

0  Route,  Speed,  Time 
0  Update  Points 
0  Target  Alrea/Attack 
Planning 
0  Deconfliction 
o  Formation  planning 
(flightpath  separation, 
timing) 

0  ECM  Planning 


Fig.  5 


Data  Link  Processing  Module  for  Crew  Assistant 


Inputs: 

o  MIDS  Network  input  Data: 
Threatpoattlan,-ty^,-status 
Targetposltlon  .-type  .-status 
ATO  Attack/Formatlon  Info 
RadRefPte.RAC-Areos. 
Command/Formation  Data 

Data  Link  Message  Display 

input: 

standard  Nato  MessageFormat: 
-ATO-,  Command-. 

-Sensor-,  Formation-, 
-Targot-.Threat-  and 
-Navigation  Data 


Output 


Standeu’d  Nato  MessagaFormat: 
-Sensordata.Farmatlon-. 
-Target-  and  Threat-Data 


Functions: 

o  Unpacking,  decoding  and 
selection  of  incomming  data 

o  Data  Fusion  of  DB  •  and 
MIDS  -  Data,  Sensortuslon 
with  ownship  data,  data 
base  update 

o  Command  and  formation 
Ink)  extraction  and  update 
of  mission  planning  and 
situation  assessment 
(COMAO  Connectivity) 

o  Outgoing  information  sel¬ 
ection,  coding  and  packing 

o  Operation  of  data  link  tran¬ 
smit  and  receive  module 


Uaod  Data  Baaea: 

o  Long/Lat/AHttude  Terrain  DB 
Navigation-.  RRP-,RAC-DB 

o  Threat  DB  (  radar  range.tlre- 
Ing  range,  ECM  bumthrough 
range,  chaff  effectivtty,  SAM 
maneuver  enveloppe  ) 

o  Target  DB  (  position  .type.sta- 
tu8,attack-conidor, -profile, 
terminal  radar  update  points) 


o  Network  addressee,  codes 


o  Air  Space  Order  (Nofly) 


o  Support  (Jam  Corridor)  DB. 
Countermeasures  DB 


a  Weather  DB 


Outputs: 

o  MIDS  Network  Output  Data 
Threatposltlon.-typo.-status 
Targetpositlon.-typo.-status 
ATO  Attack/Foimatlon  info 
Command/Formation  Data 


Fig.  6 
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Human-Electronic  Crew-Test  and  Evaluation  Issues 

Wg  Cdr  Ian  Burrett,  OC  Heavy  Aircraft  Test  Squadron,  DERA  DTEO, 

Boscombe  Down,  UK 

This  paper  will  discuss  the  issues  arising  for  aircraft  test  and  evaluation  (T&E)  from  the 
concepts  of  the  Human  Electronic  Crew.  The  role  of  T&E  in  aircraft  acceptance  and  the 
problems  of  man-in-the-loop  assessment  will  be  outlined.  Current  T&E  methods  for 
evaluating  aircrew  systems  will  be  described,  in  particular  the  approaches  used  for  assessing 
crew  workload  and  performance.  HEC  T&E  issues  associated  with  human-computer 
interaction  have  beginnings  in  the  introduction  of  digital  avionics  and  automation  of  mission 
systems,  and  associated  new  cockpit  technologies,  from  the  late  1960s  and  early  1970s.  These 
technologies  have  dramatically  changed  the  nature  of  aircrew  tasks.  Beginning  with  the 
Harrier  GRl  and  Tornado  Mkl  programmes,  through  upgrades  such  as  to  Jaguar,  to  the 
current  programmes  such  as  Merlin,  attach  Helicopter,  and  Eurofighter,  aircrew  tasks  have 
become  more  about  thinking  than  doing,  md  more  cognitive  than  physical  in  nature.  The  need 
to  assess  aircrew  mental  workload,  teamworking  performance,  and  the  useability  of  cockpit 
systems,  in  a  scientifically  valid,  quantifiable  and  reliable  manner,  has  become  an  increasingly 
important  issue  for  aircraft  T&E;  DERA  DTEO  have  had  recent  experience  with  glass  cockpit 
automation  T&E  issues,  through  the  C130J  programme,  and  arising  out  of  die  ESR  Future 
Large  Aircraft  project.  The  T&E  issues  and  human  factors  lessons  learnt  associated  with  the 
use  of  advanced  cockpit  automation  technology,  arising  from  these  programmes  will  be  the 
basis  of  this  paper. 


185 


TfflS  PAGE  INTENTIONALLY  LEFT  BLANK 


186 


Automated  support  for  Command  and  Control  (C^: 
Experience  from  a  naval  technology  demonstrator 


R  E  King,  Dr  J  A  H  Miles,  Lt  Cdr  S  D  Molyneux,  Sea  Systems,  DERA  Portsdown,  Hants  P017  6AD,  UK. 


Any  views  expressed  are  those  of  the  authors 
and  do  not  necessarily  represent  those  of  the 
Department/HM  Government. 

Abstract 


Modern  warships  are  increasingly  likely  to  be 
engaged  in  littoral  warfare  and  combined 
operations  with  other  ships,  aircraft  and  land 
forces,  employing  sophisticated  weapons  and 
techniques  intended  to  maximise  effectiveness 
and  minimise  own-force  casualties,  expenditure 
of  munitions  and  collateral  damage.  Access  to 
information  is  vital  for  the  detailed  situation 
awareness  and  precision  demanded  by  such 
complex  scenarios. 

The  command  must  also  be  able  to  react 
rapidly  when  faced  with  fast-moving  stealthy 
threats  employing  sophisticated  deception  and 
high-speed,  high-accuracy  weapons.  In 
addition  to  sea-borne  threats,  a  wide  range  of 
land-based  weapons  may  be  deployed  against 
warships  close  in  shore. 

The  consequent  development  of  high- 
bandwidth  communications  for  the  exchange  of 
intelligence  and  force  sensor  information,  and 
increasingly  sophisticated  on-board  sensors, 
have  resulted  in  an  increase  in  the  information 
available  for  Command  and  Control. 

High  data  rates  demand  rapid  processing  if  the 
command  is  not  to  be  overwhelmed  by  data. 
Because  of  this,  high  levels  of  automated 
support  are  already  implicit  in  the  requirements 
for  future  UK  Royal  Navy  combat  management 
systems. 

Technology  demonstration  is  a  key  tool  with 
which  to  research  the  provision  of  sophisticated 
and  effective  automated  support  for  command 
decision  making.  As  well  as  providing  real-time 
animation  of  the  algorithms  and  techniques  of 
automation,  often  in  an  operational  context, 
technology  demonstrators  also  address  critical 
technical  issues  involved  in  procuring  and 


integrating  components,  both  hardware  and 
software,  of  future  systems. 

The  UK  Defence  Evaluation  and  Research 
Agency  (DERA)  has  developed  the  Data  Fusion 
Technology  Demonstrator  System  (DFTDS)  for 
combat  management  systems  research.  The 
DFTDS  is  a  large-scale  prototype  knowledge- 
based  system  for  real-time  data  fusion.  It 
provides  automated  support  for  the  naval 
Command  and  Control  functions  of  tactical 
picture  compilation,  situation  assessment  and 
reactive  resource  allocation  by  encapsulating 
these  functions  in  software  modules  employing 
rule-based  artificial  intelligence  techniques. 
The  DFTDS  is  enabling  DERA  to: 

•  identify  combat  management  functions  that 
can  benefit  from  automation,  distinguishing 
these  from  others  which  may  be  high  risk, 
or  where  the  benefits  of  automation  are 
less  clear; 

•  develop  measures  of  performance  for 
automated  systems; 

•  successfully  apply  knowledge-based 
systems  to  provide  real-time  automated 
assistance; 

•  fuse  live  data  at  sea  in  an  operational 
environment; 

•  investigate  sensor  and  other  data  interfaces 
required  to  interact  with  automated  combat 
systems; 

•  investigate  the  use  of  databases  in  data 
fusion; 

•  implement  a  flexible,  reconfigurable  MMI; 

•  study  novel  forms  of  data  input  such  as 
voice  input; 

•  study  changes  in  the  role  and 
responsibilities  of  operators. 

Introduction 

The  principal  enabling  technology  to  be 
discussed  is  KBS  (Knowledge-Based  Systems) 
and  its  application  to  the  automation  of 
functions  within  an  afloat  Combat  Management 
System  (CMS). 
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This  paper  provides  a  brief  description  of  the 
DFTDS,  followed  by  a  discussion  of  lessons 
learned  in  providing  automated  support  for 
shipborne  Command  and  Control,  based  on 
experience  in  the  laboratory  and  at  sea  in  the 
users’  environment. 

Data  fusion  at  sea 

An  important  function  of  Command  and  Control 
at  sea  is  the  extraction  of  information  from  all 
available  data  in  order  to  describe  the  real 
world  tactical  environment  of  the  ship.  In 
existing  systems,  this  is  first  accomplished  by 
manual  compilation  of  the  tactical  picture, 
consisting  of  vehicles  (ships,  aircraft,  missiles 
etc.)  operating  in  a  tactical  context. 

Vehicles  in  the  tactical  picture  may  be 
supported  by  a  variety  of  evidence  from 
different  sources,  i.e.  the  data  is  fused.  The 
identity,  position  and  intent  of  vehicles  is  then 
used  to  reason  about  the  tactical  situation  and 
to  determine  an  appropriate  response.  In 
modem  maritime  warfare,  with  anti-ship 
missiles  capable  of  speeds  up  to  1000m/s, 
Command  and  Control  functions  have  become 
increasingly  time  critical. 

DFTDS  functionality 

DFTDS  takes  in  real-time  data  (e.g.  position, 
course  and  speed)  from  own  ship’s  sensors 
(e.g.  radars,  sonars,  IFF,  ESM);  other  real-time 
inputs  are  ship’s  position  data  and  operator 
input  entered  manually  via  keyboard,  tracker 
ball,  and  an  experimental  voice  input  system 
used  to  enter  identity  overrides.  DFTDS  also 
takes  in  near-real  time  data  from  other  ships’ 
and  aircrafts’  sensors  via  datalinks.  Non-real 
time  frequently  changing  data  such  as  IFF 
codes,  plans,  intelligence,  signals  etc.  are 
entered  by  the  operator  into  the  DFTDS’ 
operational  database.  Fixed  data  such  as 
shore  line  and  air  lane  positions  are  available  to 
the  DFTDS  via  its  geographic  database.  A 
KBS,  implemented  in  the  data  fusion  module, 
fuses  this  data  and  presents  it  to  the  operators 
in  a  tactical  picture. 


Figure  1:  DFTDS  block  diagram 

Further  KBS  modules  use  the  fused  tactical 
picture  to  assist  the  command  in  assessing  the 
situation  (the  situation  assessment  module) 
and  recommending  actions  to  be  taken  (the 
resource  allocation  module).  These  modules 
are  less  mature  than  the  data  fusion  module, 
hence  this  discussion  will  concentrate  on  data 
fusion  for  tactical  picture  compilation. 

Data  fusion  mechanisms 

DFTDS  is  capable  of  fusing  a  variety  of  data 
types  (reals,  integers,  enumerations, 
associations)  from  a  numbers  of  sources  (real¬ 
time,  near  real-time,  non-real  time  changing, 
and  fixed).  An  approach  to  data  fusion  that 
considers  all  possible  combinations  is 
untenable  due  the  resultant  combinatorial 
explosion  of  possibilities:  modern  sensors  are 
capable  of  holding  tracks  on  hundreds, 
sometimes  thousands,  of  objects 
simultaneously:  the  capacity  of  tactical  data 
links  is  continually  improving;  increasingly 
responsive  database  management  systems  are 
capable  of  providing  large  amounts  of  detailed 
data  within  very  narrow  time  constraints. 

DFTDS  overcomes  these  problems  by  adopting 
a  hierarchical  approach  to  data  fusion.  In 
summary,  this  involves  a  two  distinct  stages, 
correlation  and  combination. 

In  the  first  stage,  data  from  similar  sensors  is 
fused,  e.g.  an  own-ship  target  indication  radar 
track  with  an  own-ship  navigation  radar  track; 
similarly,  data  from  the  own-ship  bow-mounted 
sonar  is  fused  with  data  from  the  own-ship 
towed  array  sonar.  In  order  to  fuse,  tracks 
must  meet  a  number  of  track  correlation  criteria 
(e.g.  environment,  speed,  position). 


The  result  of  this  fusion  of  data  from  similar 
sensors  is  a  representation  intermediate 
between  tracks  and  vehicles,  referred  to  as  a 
multi-track.  Multi-tracks  are  of  different  types 
according  to  the  source  of  their  associated 
tracks.  Multi-tracks  of  different  types  that 
satisfy  the  multi-track  correlation  criteria  are 
then  fused  to  create  vehicles.  Figure  2  below 
illustrates  the  process,  where  V  is  the  vehicle, 
C  is  a  confirmed  correlation,  MT  is  a  multi¬ 
track,  and  T  is  a  track: 


Figure  2:  Data  fusion  hierarchy 

Correlations  are  continually  re-assessed  as 
tracks  are  updated.  Updates  that  fail  to  satisfy 
the  correlation  criteria,  together  with  new 
tracks,  start  the  correlation  process  over  again. 
The  result  of  the  first  stage  of  the  fusion 
process  is  to  populate  the  tactical  picture  with 
vehicles,  either  as  point  or  bearing  only. 

The  second  stage  of  the  fusion  process 
combines  track  information  with  collateral 
information  such  as  IFF,  air  lane  status  (e.g.  in 
lane,  crossing  lane,  out  of  lane  -  assessed  by 
reference  to  the  geographic  database), 
behaviour  assessment  (e.g.  fast  closing 
contact),  data  link  assessed  allegiance,  emitter 
identity,  etc.  The  result  is  to  ascribe  an 
allegiance  (friendly,  neutral,  hostile,  unknown) 
and  an  identity  to  vehicles  in  the  tactical  picture. 
The  allegiance  assessment  is  tagged  as 
possible,  probable  or  confirmed  by 
consideration  of  the  weight  of  evidence  and  the 
war  state,  e.g.  a  vehicle  considered  ‘probable 
neutral’  in  peace  time  may  be  considered 
‘possible  hostile’  in  time  of  war. 

The  tactical  picture  is  now  available  for  display 
to  the  operator,  and  is  also  passed  to  the 
situation  assessment  module  to  identify 
groupings  and  threats.  Threats  are  then 


passed  to  the  resource  allocation  module  to 
recommend  appropriate  actions  to  the  operator. 

It  should  be  noted  that  the  correlation  criteria 
and  weights  of  evidence  permit  a  degree  of 
flexible  parameterisation  of  the  KBS  reasoning 
process.  Changes  to  parameters  may  be 
acheived  at  system  start-up,  or  at  run-time, 
allowing  the  operator  to  change  the  response  of 
the  KBS. 

Tactical  picture  display 

DFTDS  represents  the  first  attempt  in  a  UK  RN 
Operations  Room  to  use  colour  and  a  windows- 
type  interface  in  an  electronic  tactical  picture. 

Display  of  multiple  windows  is  possible,  and 
windows  may  be  moved  and  sized  dynamically. 
This  flexibility  means  that  terminals  may  be 
configured  during  system  run-time  to  suit 
particular  warfare  specialisms,  e.g.  air,  sub¬ 
surface  etc.,  reflecting  RN  doctrine  for  the 
organisation  of  Operations  Room  teams. 

As  much  information  as  possible  has  been 
included  in  the  non-standard  symbology:  shape 
is  used  to  represent  environment,  e.g.  an 
aircraft-shape  symbol  for  fast  air  vehicles; 
orientation  for  assessed  heading;  colour  for 
allegiance,  e.g.  red  for  hostile.  Symbols  filled 
with  colour  are  held  live,  hollow  symbols  are 
those  being  dead-reckoned.  Land  and  sea  are 
represented  in  plan  form,  coloured  green  and 
blue  respectively. 

Data  used  in  the  fusion  process  such  as 
airlanes,  sector  screen  plans,  navigation 
features  etc.  is  also  directly  available  to  the 
operator  in  the  tactical  picture  as  graphical 
overlays,  further  enhancing  operator 
appreciation  of  the  tactical  situation. 

Seiecting  an  object  in  the  tactical  picture  by 
means  of  the  tracker  ball  or  mouse  brings  up 
further  windows  containing  a  mix  of  graphics 
and  text  that  provide  information  about  the 
vehicle  and  its  supporting  tracks.  Selection  of 
fields  in  these  windows,  such  as  the  assessed 
allegiance,  brings  up  further  windows  that 
explain  the  system’s  reasoning. 

The  data  fusion  process  is  made  visible  further 
by  providing  the  operator  with  graphical 
displays  at  the  track,  multi-track  and  vehicle 
levels.  Various  filters  are  available  that  allow 
the  operator  to  display  selected  sources, 
environments  and  allegiances. 
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Operator  interaction 

Although  the  operator  may  work  with  the 
vehicle-level  tactical  display  alone,  he  can,  by 
use  of  the  track  level  window  and  by  use  of 
vehicle  explanation  windows  and  other  windows 
such  a  plan  displays,  obtain  quick  access  to  the 
track  level  and  other  data  supporting  a  DFTDS 
vehicle  hypothesis. 

Manual  override  facilities  allow  the  operator  to 
correlate  and  de-correlate,  and  to  change  the 
identity  of  contacts. 

A  concise  history  of  the  data  fusion  associated 
with  a  vehicle  is  also  available  to  him.  This 
visibility  of  the  data  fusion  process  is  facilitated 
by  KBS:  it  would  not,  for  example,  be  possible 
to  provide  the  same  intuitive  explanations  if 
neural  nets  were  employed. 

System  size 

The  DFTDS  consists  of  around  350,000  lines  of 
Ada83  code,  makes  extensive  use  of 
commercial-off-the-shelf  (COTS)  hardware  and 
software,  and  has  approximately  900  rules,  of 
varying  complexity,  in  its  three  knowledge 
bases.  It  has  taken  about  200  man-years  to 
construct. 

The  sea-going  DFTDS,  dating  from  1991,  was 
hosted  on  DEC  VaxA/MS  and  Sybase  RDBMS. 
Apart  from  basic  shock-mounting  of  the  rack 
cabinets,  the  system  was  not  ruggedised  in  any 
way.  Colour  graphics  were  provided  by  five 
Sigmex  colour  graphics  workstations.  At  sea, 
the  DFTDS  COTS  equipment  proved  to  be 
reliable  and  rugged.  Recently,  in  the 
laboratory,  the  software  has  been  re-hosted  on 
DEC  Alpha/OSF,  and  work  is  in  place  to  move 
to  an  open  UNIX  system. 

Knowledge  representation 

KBS  is  a  technology  which,  in  principle,  enables 
any  available  (expert?)  knowledge  for  solving 
specific  problems  to  be  engineered  in  software. 
It  is  also  a  methodology  which  prescribes  a 
prototyping  approach  to  development,  a 
programming  style  emphasising  visibility  of  the 
encapsulated  knowledge  to  non-computer 
specialists  and  a  user  interface  providing 
explanations  of  the  machine  reasoning.  One 
may  regard  KBS  as  a  software  engineering 
technology  for  complex  human/machine 
software  intensive  systems;  it  is  thus  well  suited 
to  complex  Command  and  Control  functions  in 


which  data  is  not  simply  stored  or  displayed  but 
is  combined,  transformed  and  interpreted  using 
scientific  and  expert  knowledge  of  the 
application  domain.  It  enables  complex 
automated  support  functions  to  be  implemented 
as  shown  by  the  DFTDS. 

The  main  limitation  of  KBS  is  that  it  is  restricted 
to  fixed  knowledge  of  a  specific  nature  -  it  is  not 
to  be  regarded  as  intelligent  in  the  human 
sense.  In  particular,  although  the  reasoning 
process  may  be  adapted  by  changes  to  rule 
parameters,  it  has  no  learning  capability.  The 
adaptability  that  parameterisation  provides 
enables  the  developer  to  fine-tune  the  KBS, 
however  the  role  of  such  adaptability  in  the  field 
has  not  been  researched.  If  a  Command 
System  modified  its  rules  from  experience  of 
many  different  situations  and  uncertainties, 
different  ships  would  end  up  with  different  sets 
of  rules.  Furthermore  some  of  these  rules  may 
have  been  modified  as  a  result  of  inappropriate 
lessons  being  learnt  from  the  many  different 
situations  and  uncertainties. 

Given  that  the  knowledge  is  static  then  the 
problem  that  the  KBS  solves  must  also  be 
static,  or  at  least  very  slowly  changing.  It  must 
be  a  problem  that  is  well  understood  and  which 
will  persist  for  a  considerable  time  -  certainly 
much  longer  than  it  takes  to  develop  the  KBS. 
Development  time-scales  longer  than  a  few 
years  are  unlikely  to  be  useful  in  the  Command 
and  Control  application  domain  and  even  when 
developed,  the  KBS  will  have  to  be  continuously 
maintained  in  order  to  keep  it  up  to  date. 

It  might  appear  that  a  learning  or  self  adapting 
capability  would  overcome  the  fixed  knowledge 
limitation  and  this  is  being  actively  explored  in 
the  research  community.  Learning  systems 
are  not  yet  practical  except  in  very  restricted 
domains,  and  may  not  be  appropriate  for  the 
Command  and  Control  application,  where 
visibility  of  the  reasoning  is  crucial  to  user 
confidence. 

It  is  tempting  to  look  at  Command  and  Control 
systems  as  performing  a  number  of  objective, 
describable,  predetermined  functions  and 
believing  that  each  can  arbitrarily  ascribed  to 
human  or  machine.  However  it  should  be 
recognised  that  although  a  set  of  necessary 
functions  can  be  described  and  indeed 
supported  by  KBS  to  a  considerable  degree  (as 
proven  by  DFTDS),  there  is  still  a  necessary 
and  essentially  subjective  human  element 
which  must  not  be  excluded. 
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Knowledge  elicitation 

Builders  of  a  KBS  will  meet  subjectiveness 
early  in  the  knowledge  elicitation  process.  In 
DFTDS,  one  example  occurred  when 
attempting  to  formulate  rules  for  assessing 
whether  a  vehicle  is  friendly,  neutral,  hostile  etc. 
Five  different  people,  each  of  whom  had  had 
recent  command  experience,  were  asked  what 
assessment  they  would  give  in  a  tactical 
example.  Perhaps  not  surprisingly,  there  were 
three  different  answers.  This  and  other  similar 
experience  has  lead  us  to  deduce  that  it  is 
impossible  to  put  into  a  KBS  reasoning  which 
will  reflect  every  human’s  thinking  when 
judgement  is  called  for  (automation  of 
reasoning  processes  cannot  universally  satisfy 
judgmental  criteria). 

Data  representation 

The  accuracy  and  reliability  of  data  fusion  is 
crucially  dependent  on  the  quality  of  information 
available  from  sensor  and  communications 
systems.  In  the  past  the  process  of  extracting 
information  and  control  of  quality  was  entirely  a 
human  responsibility  but  now  we  are  beginning 
to  rely  increasingly  on  automatic  or  semi¬ 
automatic  information  extraction. 

Unfortunately,  though  sophisticated,  the 
technology  for  this  automation  still  has  many 
failings  especially  in  controlling  quality  of 
output.  In  our  experience  this  issue  has  been 
the  biggest  limiting  factor  in  the  performance  of 
KBS  functions.  The  quality  of  information  can 
be  highly  inconsistent,  and  few  if  any  of  today’s 
sensors  provide  adequate  measures  of  the 
confidence  which  may  be  placed  on  data  they 
report. 

The  performance  of  data  fusion  at  sea  can  be 
degraded  by  poor  position  registration  between 
ships  and  aircraft  participating  in  the  tactical 
datalink.  This  can  lead  to  the  appearance  in 
the  tactical  picture,  for  example,  of  two  air  raids 
-  one  held  by  own-ship’s  radar,  the  other 
reported  over  the  datalink  -  when  in  fact  there  is 
only  one.  Algorithms  were  developed  in 
DFTDS  to  automatically  correct  for  such 
position  errors,  but  in  some  cases  operator 
intervention  was  required  to  initially  establish 
correct  registration.  Operator  intuition  or 
appreciation  e.g.  ‘I  know  the  link  from  HMS 
UNKNOWN  is  bad  today  and  I  make  due 
allowance’  were  of  great  value  in  such 
situations. 


Simulation  of  improved  sensor  inputs  has 
shown  that  satisfactory  data  fusion 
performance  is  only  likely  to  be  reached  with 
improved  sensor  processing.  Therefore  there 
is  a  practical  limitation  on  any  Command  and 
Control  system  imposed  by  the  practical 
limitations  on  their  data  sources.  This  limitation 
has  been  highlighted  by  the  DFTDS  trials 
programme.  Although  the  DFTDS  has 
demonstrated  that  KBS  are  capable  of  usefully 
reasoning  with  data  of  widely  differing  types, 
their  inherent  lack  of  flexibility  means  that  they 
are  prone  to  error  if  real  world  information  is 
represented  to  them  in  a  mis-leading  way. 

Data  elicitation 

All  military  Command  and  Control  systems 
require  information  from  signalled  messages: 
for  example,  the  position  where  a  fighter  aircraft 
is  to  patrol  is  promulgated  in  a  signal  and  this 
information  can  be  used  in  the  identity 
hypothesis. 

Signal  data  must,  however,  be  entered  in  a  rigid 
format  for  a  computer  to  be  able  to  parse  it  and 
then  make  use  of  it.  In  practice,  entering 
signalled  messages  like  this  by  hand  was 
known  to  be  time  consuming  and  error  prone. 
The  possibility  of  entering  signalled  messages 
automatically  into  DFTDS  was  considered  but 
when  relevant  signals  from  an  exercise  were 
analysed,  it  was  found  that  none  was  amenable 
to  automatic  interpretation.  In  each  case  the 
format  laid  down  in  the  relevant  NATO 
publication  was  not  strictly  adhered  to,  because 
humans  can  interpret  and  so  do  not  need  to 
follow  rigid  rules;  indeed,  if  they  did  they  would 
find  it  very  restricting. 

One  answer  to  this  is  to  enforce  on  senders  of 
messages  compliance  to  formats.  However,  if 
this  were  to  be  done,  the  sender  would  not  be 
able  to  communicate  everything  that  he  felt  he 
wanted  to;  for  example,  he  might  want  to 
amplify  his  thinking  behind  ordering  a  ship  to  a 
specific  position,  or  indeed  may  wish  to 
promulgate  policies  for  positioning  rather  than 
specific  positions. 

Therefore  from  our  practical  experience  we 
deduce  that  there  is  a  limit  to  how  far  one  can 
automate  communication. 

Limits  to  automation 

Command  must  take  responsibility  for  all 
actions  taken,  highest  priority  being  given  to 


191 


those  which  are  life-threatening:  this  is  the  point 
of  a  Command  and  Control  system.  In  some 
cases  prior  authority  may  be  given  for  an 
automatic  response  to  specific  events,  e.g. 
point  defence,  jamming  ploys,  but  these  must 
operate  in  a  strictly  limited  domain  and  under 
the  simplest  of  rules  in  order  to  avoid 
unpredictable  behaviour. 

The  ability  of  humans  to  learn  and  solve 
problems  is  unmatched  by  any  machine 
technique.  Also,  the  ability  of  humans  to  ‘get 
into  the  mind  of  an  enemy’  i.e.  intuitive 
reasoning,  is  a  uniquely  human  characteristic. 
These  abilities  must  be  recognised  as  most 
important  in  the  unpredictable  scenarios  of 
modern  warfare  and  must  not  be  inhibited  by 
any  automated  support  functions. 

The  element  of  surprise,  often  by  the 
deployment  of  new  or  ‘crazy’  tactics,  has  been 
crucial  in  any  number  of  celebrated  military 
victories,  e.g.  Nelson’s  defeat  of  the  French  at 
Trafalgar,  Napoleon’s  defeat  of  the  Austrians  at 
Austerlitz.  It  is  extremely  unlikely  that  a  KBS, 
essentially  a  conservative  doctrine-based 
system,  would  recommend  the  tactics 
employed  to  secure  these  victories.  The 
converse  of  this  is  that  a  tactical  response 
encoded  in  a  KBS  could  give  an  advantage  to 
an  enemy  who  gained  access  to  it. 

The  role  of  the  expert 

KBS  require  expertise  which  is  in  very  short 
supply  and  distributed  among  a  number  of 
experts.  Unless  some  way  is  found  to  retain 
this  expertise  then  it  may  happen  that 
knowledge  will  be  lost  and  systems  of  the  future 
become  less  capable. 

Expertise  which  is  regularly  applied  and  for 
which  performance  metrics  can  be  regularly 
monitored  is  valid  to  acquire  for  a  KBS.  In 
Command  and  Control  however,  much  of  the 
‘expert’  knowledge  is  rarely  put  into  practice 
and  is  consequently  very  difficult  to  validate; 
this  could  explain  why  experts  often  differ  or  it 
could  indicate  that  their  knowledge  has  a 
subjective  component.  Such  knowledge  would 
be  dangerous  to  use  for  a  KBS  and  will 
ultimately  limit  the  scope  of  machine  support  - 
though  this  is  a  proper  limitation  in  the  sense 
that  the  strength  of  machines  is  in  objective 
reasoning  and  that  subjective  reasoning  is  the 
province  of  humans. 


Some  expert  knowledge  can  be  very  hazardous 
to  codify  in  a  KBS.  An  example  here  is  that  in 
the  standard  RN  surface-ship  training  at 
Portland,  operators  knew  that  on  Thursdays 
the  air  threat  came  from  the  west  and  so  they 
would  assume  that  any  object  approaching 
from  the  west  at  high  speed  was  an  enemy 
without  any  further  evidence.  DFTDS  does  not 
have  this  knowledge,  which  is  based  on 
experience,  and  although  it  could  be 
incorporated  by  reference  to  the  time,  day  of 
the  week  and  ship’s  position,  the  dangers  of 
attempting  to  incorporate  it  are  obvious.  Note 
that  this  is  just  the  sort  of  conditions  where  a 
neural  net  approach  to  data  fusion  could  yield 
similarly  dangerous  results,  with  less  obvious 
visibility  of  trouble  to  come. 

Trust 

A  problem  with  using  a  KBS  to  fuse  data  and 
present  an  assessed  picture,  is  that  for  the 
operator  to  accept  this  highest  level  picture 
implies  complete  trust  on  his  part  in  the  data 
fusion  process  which  has  created  it.  Will,  or 
should,  humans  totally  trust  a  computer? 

Assuming  that  the  answer  to  that  is  ‘no’,  a  KBS 
must  have  some  way  of  explaining  its 
reasoning  process  to  those  who  use  its  output. 
Here  is  a  significant  bottleneck  in  the  Human 
Computer  Interface  (HCI).  Even  using  modern 
colour  graphics  to  convey  explanations  of  the 
system  operation  to  the  users  there  is  so  much 
information  and  reasoning  being  performed  by 
the  machine  in  real-time  that  a  human  cannot 
keep  up.  It  is  difficult  to  present  all  the 
reasoning  behind  a  hypothesis,  and  particularly 
difficult  to  represent  levels  of  confidence,  to  an 
operator,  and  actually  some  of  it  requires  a 
depth  of  knowledge  beyond  that  of  the  average 
operator. 

Interestingly,  our  experience  at  sea  is  that 
explanations  are  not  sought  as  much  as 
anticipated,  partly  because  obtaining 
explanations  can  be  too  slow  for  fast  moving 
situations,  partly  because  operators  sometimes 
find  it  difficult  to  understand  the  explanation, 
and  partly  as  ‘It’s  obvious  when  it’s  wrong,  so  I 
don’t  need  an  explanation’.  There  is  certainly 
danger  in  users  simply  disbelieving  the  data 
before  them.  None-the-less,  an  explanation 
facility  is  considered  by  users  as  an  essential 
feature  and  has  proven  invaluable  in  system 
development. 
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Procurement  Policy 

The  current  approach  to  defence  procurement 
does  not  seem  to  have  had  much  success 
when  complex  software  is  involved.  Although 
cost  can  be  controlled,  time-scale  and  the 
ability  of  the  system  to  meet  operational 
requirements  usually  suffer.  This  situation  will 
be  exacerbated  by  attempting  to  introduce 
higher  levels  of  automated  support  because 
these  levels  are  much  more  sensitive  to  the 
operational  environment.  Our  attempts  to  find 
KBS  systems  which  have  been  formally 
procured  have  failed  -  the  practice  every  time 
seems  to  be  a  series  of  prototypes  and  those 
that  are  successful  require  constant  adaptation 
-  certainly  in  complex  problem  domains. 

The  incremental  approach  used  for  building  the 
DFTDS  in  fixed  time  and  cost  stages  has 
worked  well  but  it  is  poorly  matched  to  current 
military  procurement  strategies.  Unless 
knowledge  captured  in  this  way  can  be 
maintained  and  enhanced  on  a  continuous 
basis,  and  exploited  as  widely  as  possible  then 
the  procurement  process  will  place  an  artificial 
limit  on  the  use  of  technology. 

Summary 

DFTDS  is  of  sufficient  scale  to  provide  realistic 
lessons  for  future  RN  Command  and  Control 
procurements.  This  is  what  is  expected  of  a 
technology  demonstrator  programme,  but  is  not 
easy  to  achieve  in  the  Command  and  Control 
domain  because  of  the  software  complexity 
involved. 

In  brief,  the  achievements  of  the  programme  so 
far  are: 

•  By  adapting  KBS  techniques  we  have  been 
able  to  build  a  system  which  runs  in  real¬ 
time  and  with  sufficient  capacity  to  handle 
real  data  volumes  obtained  at  sea. 

•  We  have  shown  the  potential  value  of  using 
commercial  off  the  shelf  hardware  and 
software  for  reduced  acquisition  and  owner¬ 
ship  costs. 

•  Incremental  development  with  user 
involvement  and  feedback  from  trials  has 
been  a  great  success. 

•  We  have  established  a  baseline  of  data 
fusion  performance  with  existing  inputs  and 
are  now  in  the  process  of  predicting,  by 
demonstration,  what  level  of  performance 
can  be  achieved  with  sensor 
improvements. 


•  The  display  symbology  and  HCI  generally 
have  been  a  great  success,  providing 
operators  with  an  instant  grasp  of  the 
tactical  situation  with  a  single  glance  at  the 
screen. 

•  We  have  established  a  large  amount  of 
detailed  knowledge  concerning  the 
capabilities  and  limitations  of  the 
technology  and  shipborne  environment. 

The  following  lessons  have  been  learned: 

•  KBS  technology  allows  complex  knowledge 
to  be  mechanised  but  that  knowledge  is 
fixed  once  it  has  been  engineered. 
Adaptive  techniques  are  becoming 
available  but  it  is  not  clear  how  to  use  them 
in  a  complex  Command  and  Control 
domain  without  a  system  learning 
inappropriately. 

•  Some  knowledge  is  subjective  and  should 
not  be  mechanised  -  the  boundary  of 
subjectivity  is  not  always  easy  to  define. 

•  The  human  computer  interface  can  be  a 
bottleneck  in  terms  of  machines  being  able 
to  understand  normal  human 
communication. 

•  Some  knowledge  is  difficult  to  acquire  and 
validate  owing  to  its  rarity  and  infrequent 
use. 

•  Automated  support  functions  rely  on  quality 

controlled  data  inputs.  These  are  not 
available  from  current 

sensor/communications  systems. 

•  There  is  more  information  than  a  human 
can  keep  pace  with  -  automation  adds  extra 
layers  to  the  data  available  all  of  which  may 
be  needed  to  provide  users  with  the 
confidence  they  need  to  be  responsible  for 
actions. 

Future  developments 

Much  work  remains  to  be  done  to  further 
evaluate  data  fusion  for  tactical  picture 
compilation  and  to  develop  automated  support 
for  situation  assessment  and  resource 
allocation. 

Data  fusion  at  the  force  level,  as  well  as  the 
single  ship  level,  will  be  studied  in  our  on-going 
research  programme. 

DFTDS  in  its  open  systems/open  standards 
form  (now  dubbed  CMISE  -  the  Combat 
Management  Integration  Support  Environment) 
will  provide  a  core  component  of  DERA’s 
programme  of  technology  demonstration  and 
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research  to  evaluate  potential  solutions  to 
future  combat  system  requirements.  It  will  also 
play  a  vital  role  to  de-risk  the  development  of 
functional  and  non-functional  requirements  on 
which  to  base  the  procurement  of  future 
combat  systems. 
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1  Summary 

The  physiological  state  of  the  crew  of  the  aircraft 
has  been  ignored  in  the  development  of  Human- 
Electronic  Crew  systems.  Environmental  stresses 
such  as  acceleration,  altitude,  fatigue,  and  thermal 
load  can  challenge  the  physiological  capability  of  the 
crew  such  that  cognitive  function  is  degraded  or  even 
eliminated  as  in  the  extreme  case  of  Gz  induced  loss 
of  consciousness.  This  paper  describes  both  the  de¬ 
velopment  of  first-principle  and  data  driven  models 
of  the  physiological  and  cognitive  changes  that  occur 
in  the  pilot  exposed  to  the  tactical  environment  and 
the  AI  techniques  under  development  for  both  mon¬ 
itoring  and  incorporating  the  physiological  and  cog¬ 
nitive  state  of  the  pilot  into  human-electronic  crew 
systems.  Such  a  capability  will  allow  the  optimiza¬ 
tion  of  the  electronic  component  of  the  crew  with 
respect  to  the  physiological  and  neurologiceil  state 
of  the  pilot. 


2  Introduction 

In  almost  all  discussions  of  human-electronic  crew 
systems  there  has  been  little  mention  as  to  to 
whether  or  not  the  human  crew  has  the  physiolog¬ 
ical  “right  stuff”  -  i.e.,  is  the  human  crew  member 
capable  of  functioning  at  an  optimal  level  to  interact 
with  the  electronic  components  of  the  system?  To 
date  the  concept  of  predicting  and/or  monitoring  the 
physiological  capability  and/or  integrating  the  air¬ 
crew  life  support  system  into  the  human-electronic 
crew  has  been,  with  few  exceptions  [10,  18],  been 
ignored. 

Environmental  stresses  such  as  acceleration,  alti¬ 
tude,  and  thermal  load  can  challenge  the  physio¬ 
logical  capability  of  the  crew  such  that  sensory  and 
cognitive  function  is  degraded  as  in  the  case  of  phys¬ 
ical  or  mental  fatigue  or  even  eliminated,  as  in  the 
case  of  Gz  induced  loss  of  consciousness. 

Models  of  the  crew  can  be  developed  from  first  prin¬ 
ciples  and  empirical  data  which  can  provide  a  state 


representation  of  the  pilot.  Physiological  (and  cog¬ 
nitive)  monitoring  can  be  used  to  provide  a  real-time 
update  of  the  pilot  state  model  for  both  enhanced 
life  support  response  and  information  for  the  elec¬ 
tronic  crew-member. 


3  First  Principle  Models 

Models  of  the  various  components  of  the  human 
physiological  system  have  been  under  development 
for  a  number  of  years  [3,  21,  20,  17,  16,  6]  with  the 
majority  of  researchers  modeling  the  cardiovascu¬ 
lar,  cerebrovascular,  or  pulmonary  subsystems.  The 
models  have  increased  in  complexity  over  the  years 
as  the  computational  capability  to  solve  the  large 
number  of  differential  equations  has  also  increased 

[19]. 

In  spite  of  the  increasing  sophistication  of  these 
models  there  is  general  disappointment  in  that  few 
experimentalists  cite  them,  use  them  in  their  exper¬ 
imental  design,  or  compare  model  results  to  exper¬ 
imental  data.  The  use  of  large  numbers  of  poorly 
know  parameters  places  further  doubt  on  the  validty 
of  the  models.  A  major  drawback  to  many  of  the 
models  is  the  use  of  the  electrical  analog  representa¬ 
tion  to  describe  the  physiological  systems.  This  can 
be  misleading  when  nonlinear  physical  resistances 
and  capacitances  need  to  be  included.  In  addition 
there  is  limited  use  of  rigorous  validation,  verifica¬ 
tion,  and  sensitivity  analysis  techniques  and  almost 
no  attempt  to  validate  the  consistency,  stability,  or 
the  convergence  of  the  numerical  techniques  used  to 
solve  the  systems  of  differentia]  equations. 

The  majority  of  the  modeling  effort  has  focused  on 
clinical  applications  [19],  and  even  in  the  cases  where 
effort  is  directed  towards  acceleration  [6,  16]  or  al¬ 
titude  stress,  the  focus  has  been  on  a  single  area 
such  as  the  cardiovascular  system  or  a  subsystem 
such  as  the  cerebrovascular  system.  Almost  all  of 
the  models  developed  to  date  suffer  from  a  number 
of  inherent  limitations  in  so  far  as  their  application 
to  the  development  of  advanced  life  support  systems 
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is  concerned.  Due  to  the  complex  nature  of  the  tac¬ 
tical  environment  most  models  are  deficient  in  fully 
accounting  for  the  uniqueness  of  this  environment, 
the  complex  interactions  between  the  various  phys¬ 
iological  systems,  and  the  interactions  between  the 
physiology  and  the  life  support  systems.  For  exam¬ 
ple,  in  a  pilot  exposed  to  complex  negative  and  pos¬ 
itive  Gz  stresses  [2],  wearing  an  anti-G  suit,  positive 
pressure  breathing  mask  and  jerkin,  and  performing 
an  appropriate  anti-G  strmning  manoeuvre,  there 
will  be  an  interaction  among  the  dynamics  of  the 
cerebrovascular  system,  neuronal  and  hormonal  con¬ 
trol  systems,  muscle  function,  respiratory  mechan¬ 
ics,  respiratory  gas  exchange,  and  the  dynamics  of 
the  G-suit  and  breathing  system  pressurization  sys¬ 
tems. 

None  of  the  models  developed  to  date  can  even  in¬ 
corporate  or  model  the  anti-G  straining  manoeuvre 
( AGSM)  which  includes  isometric  contraction  of  the 
limb  muscles  and  exhalation  against  a  closed  glot¬ 
tis.  Some  models  have  incorporated  positive  pres¬ 
sure  breathing  and  anti-G  suits  [6].  The  current 
models  do  not  account  for  the  effects  of  fatigue  on 
venous  and  arterial  compliance,  for  the  decrease  in 
systemic  blood  volume  due  to  tissue  edema  with  pro¬ 
longed  +Gz  and  PPB  exposure,  or  for  the  effects  of 
changes  in  P^Oz  and  PaCOz  on  the  efiicacy  of  the 
anti-G  stradning  manoeuver, 

A  recent  paper  by  Melchior  et  ah  [15]  outlined  a 
comprehensive  firamework  for  future  efforts  in  the 
modelling  of  the  entire  cardiovascular  system,  espe- 
dally  with  respect  to  acceleration  stress.  In  order 
to  address  the  deficiencies  of  the  current  models  for 
more  global  problems  such  that  they  will  be  useful 
in  the  investigation  of  the  physiological  consequences 
of  environmental  stresses,  and  provide  a  useful  tool 
for  the  development  of  aircrew  life  support  systems, 
a  number  of  general  and  specific  enhancements  are 
required: 

» 

•  Incorporate  the  effects  of  acceleration,  alti¬ 
tude,  thermal,  exercise,  and  protective  equip¬ 
ment  induced  stresses. 

•  Incorporate  the  effects  of  positive  pressure 
breathing  (PPB)  on  chest  wall  dynamics, 
breathing  patterns,  gas  exchange,  neuronal 
control  of  breathing,  lung  baroreceptors,  ven- 
tilation/perfiision  ratios,  and  blood  chemistry, 

•  Incorporate  sensitivity  analysis  techniques  in 
the  modelling  software, 

•  Integrate  the  cardiovascular,  cardiopulmonary, 
cerebrovascular,  pulmonary,  gas  exchange,  and 


life  support  systems  models. 

•  Incorporate  thermal  regulatory  and  exercise 
responses, 

•  Integrate  lumped  parameter  models  and 
time/spatial  computational  fluid  dynamics 
models  of  the  pressure-flow  distributions  in  the 
heart  and  blood  vessels. 

•  Incorporate  the  effects  of  changes  in  PaOz  and 
PaCOz  levels  on  local  vascular  compliance,  lo¬ 
cal  blood  flow  regulation,  and  barorecptor  and 
chemoreceptor  function. 

•  Incorporate  the  effects  of  lactate,  potassium, 
and  pH  on  vascular  compliance,  local  neuro- 
transmitter  release,  and  neurotransmitter  sen¬ 
sitivity. 

•  Include  the  neuronal  and  hormonal  control  of 
vascular  compliance,  muscle  contraction,  re¬ 
cruitment  of  the  vacular  beds  (systemic  and 
cerebral),  heart  rate,  heart  contractility,  respi¬ 
ratory  rate,  airway  diameter,  and  heart  cham¬ 
ber  compliance. 

•  jIn elude  models  of  the  efficacy  of  PPB  and  G 
suit  pressure  application  to  the  thoracic  com¬ 
partment  and  abdominal/leg  vascular  system. 

•  Develop  sub-models  for  intra-  and  inter¬ 
muscular  blood  vessels,  including  the  effects 
of  muscle  and  organ  pressure/ volume  compli¬ 
ances  and  vascular  leakage. 

•  Incorporate  models  of  muscle  function  includ¬ 
ing  the  development  of  muscle  tissue  pressure. 

•  Develop  fluid  dynamical  models  of  anti-G 
suits,  PPB  garments,  and  as  well  as  the  mech- 
nical  and  fluid  dynamic  characteristics  of  anti- 
G  valves  and  breathing  regulators. 


3.1  Cerebral  perfusion 

Given  the  overall  complexity  of  the  modeling  prob¬ 
lem,  our  laboratory  has  focused  on  a  rigourous  1-D 
finite  element  model  of  the  pressures  and  flows  in 
the  heart  and  systemic  circulation  and  a  lumped  pa¬ 
rameter  model  of  the  cerebral  perfusion  of  the  head 
to  address  a  specific  problem  in  acceleration  phys¬ 
iology  -  why  does  Gz-induced  loss  of  consciousness 
(GLOC)  occur. 

The  perfusion  of  the  brain  is  dependent  on  both  the 
perfusion  pressure,  i.e.,  the  difference  between  the 
arterial  and  venous  pressures,  and  the  resistance  to 
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perfusion.  The  resistance  of  cerebral  perfusion  is  un¬ 
der  tight  control  in  the  brain  though  there  is  exten¬ 
sive  debate  as  to  the  importance  of  metabolic,  myo¬ 
genic,  or  neuronal  mechanisms  in  regulating  flow. 
Beginning  with  Akesson  [1]  and  Henry  et  al.  [7], 
it  has  been  implictly  acknowledged  that  the  Gz- 
induced  hydrostatic  gradient  should  not  affect  cere¬ 
bral  blood  flow  and  induce  GLOC,  due  to  the  pro¬ 
tection  from  the  siphon  effect. 

A  simple,  steady  state,  lumped-parameter  model  of 
cererbal  perfusion  incorporating  well-established  pa¬ 
rameters  of  the  structure  of  the  cerebral  blood  ves¬ 
sels  was  developed  (Figure  1). 


Figure  !•  Lumped  parameter  model  of  the  cere^ 
brovascular  system  used  to  determine  the  vascular 
segment  responsible  for  restricting  cerebral  blood  flow 
during  exposure  to  -hGz. 

The  governing  equations  for  the  model  are: 

•  Momentum 

(Pi  +  pngHi)  -  {Pi  +  pngHi)  =  Q 

•  Continuity 

UA  =  Q  =  const 

•  Tube  Law  of  the  blood  vessels 

=  F(A) 

•  Conservation  of  volume 

^cranial  ~  A 

This  model  demonstrated  that  the  primary  vessel 
responsible  for  regulating  blood  flow  during  posi¬ 
tive  Gz  exposure  is  the  jugular  vein  and  that  col¬ 
lapse  of  the  jugular  vein  is  responsible  for  the 


loss  of  consciousness  during  high  Gz  (Figure  2). 


-5  0  5  10 

Gz 


Figure  2.  Contribution  of  extracranial  veins  to 
blood  flow  resistance  in  the  head  during  -Gz  and  ~hGz 
exposure* 

4  Empirical  Models 

Though  the  use  of  first-principle  models  of  the  phys¬ 
iology  and  life  support  systems  can  be  useful  for 
specific  questions,  the  need  to  identify  the  exten¬ 
sive  set  of  system  parameters  still  makes  it  difficult 
to  develop  accurate  models  of  the  entire  pilot/life 
support  system.  A  more  common  approach  is  the 
development  of  empirical  models  of  the  total  sys¬ 
tem  response  to  the  various  environmental  stresses. 
With  the  use  of  both  linear  and  non-linear  analysis 
techniques,  parsimonious  models  of  the  complex  re¬ 
sponses  of  the  pilot  to  environmental  stress  can  be 
developed  that  are  rich  enough  to  characterize  the 
state  of  the  pilot. 

Models  range  from  simple  time  independent  rela¬ 
tions  of  the  cerebral  blood  flow  as  a  function  of  Gz, 
to  complex  linear  (transfer  function),  non-linear  au¬ 
toregressive  moving-aveage  (NARMAX)  models  of 
the  cardiovascular  responses  [13,  14],  and  neural 
nets.  A  typical  NARMAX  model  of  the  cerebral 
blood  flow  response  to  Gz  could  be  given  as: 


CBF{t)  =  aoCBF{t  -  1)  -h  aiCBF{t  -  1) 

+ . . .  +  anCBF{t  -  n)  -f-  hiGz(t) 

4-62^2 (t  —  2)  +  . , .  H-  bnGz{t  —  Tl) 

+coGBF(t  -  \)Gz{t) 

+  higher  order  terms* 

Given  suffificient  data  from  sufficient  pilots  over  a 
wide  range  of  enviromental  stresses,  a  comprehensive 
model  of  the  pilot’s  response  can  be  developed-  This 
is  still  insufficient  to  characterize  the  pilot  through- 
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out  the  mission. 


5  Pilot  monitoring 

No  a  priori  modeling  of  any  physiological  system 
can  account  for  all  of  the  combined  physical  stresses 
that  a  pilot  will  be  exposed  to  in  the  tactical  environ¬ 
ment.  A  more  viable  approach  is  to  have  the  most 
accurate  state  representation  of  the  nominal  pilot 
(based  on  first  principle  or  empirical  models)  stored 
in  the  aircraft  systems.  Monitoring  of  the  pilot’s 
physiological  and  cognitive  responses  can  be  used  to 
modify  the  state  model  on  a  second  by  second  basis. 
This  information  can  be  passed  to  both  the  life  sup¬ 
port  hardware  and  the  electronic  crew  member  [4], 
The  electronic  crew  member  in  turn  can  pass  infor¬ 
mation  to  the  life  support  system  to  aid  that  system 
in  providing  optimal  protection  given  the  past,  cur¬ 
rent,  and  future  state  of  the  mission. 

A  wide  range  of  physiological  signals  can  be  col¬ 
lected  on  the  cardiovascular,  neurological,  and  ther¬ 
mal  state  of  the  pilot  in  real-time.  Of  primary  inter¬ 
est  with  respect  to  acceleration  and  altitude  stress  is 
monitoring  the  delivery  of  oxygenated  blood  to  the 
brain.  Direct  measures  of  cerebral  blood  flow  and 
oxygen  levels  in  brain  tissue  would  be  ideal.  There 
has  been  significant  developments  in  near-infrared 
spectroscopy  techniques  that  do  have  the  potential 
for  non-invasive  monitoring  of  the  amount  of  oxy¬ 
genated  hemoglobin  in  the  brain  as  well  as  the  redox 
state  of  the  cytochrome  aas  molecule  of  the  respira¬ 
tory  chain.  Cerebral  blood  flow  monitoring  in  the 
codcpit  environment  is  not  yet  possible  but  the  use 
of  phased-array  doppler  probes  and  advanced  signal 
processing  techniques  may  overcome  some  of  the  dif- 
ficulites  in  monitoring  in  the  high  Gz  environment. 
A  surrogate  measure  of  oxygen  delivery  can  be  de¬ 
termined  from  monitoring  hypoxia  induced  changes 
in  electroencephalographic  (EEG)  signals,  though  it 
is  difficult  to  detect  changes  in  either  time  or  fre¬ 
quency  dependent  characteristics  of  the  EEG  signal 
during  mild  or  moderate  hypoxia. 

An  accurate  assessment  of  thermal  stress  is  one  of 
the  most  difficidt  to  obtain  (possible  with  the  use 
of  rectal  probes  but  not  acceptable  to  pilots!).  As 
the  measurement  of  thermal  stress  demonstrates,  the 
most  difficult  aspect  of  monitoring  is  the  developr 
ment  of  robust,  accurate  inexpensive  sensor  tech¬ 
nologies  that  are  both  transparent  and  acceptable 
to  the  pilot  during  pre-flight  preparation  and  in  the 
cockpit.  The  second  problem  is  the  development  of 
analog  signal  conditioning  and  digital  signal  process¬ 
ing  techniques  that  can  deliver  an  accurate  assess¬ 


ment  of  the  physiological  parameters  in  the  electri¬ 
cal  noisy  and  artifact  generating  environment  of  the 
cockpit. 

One  area  that  has  shown  significant  progress  has 
been  the  development  of  signal  processing  algo¬ 
rithms  that  correlate  changes  in  EEG  or  eye- 
movement  with  cognitive  states  and  performance  us¬ 
ing  wavelet,  time  frequency-spectral,  and  neural  net 
signal  processing  technologies  [22, 12,  11].  Accurate 
detection  and  correlation  with  performance  decre¬ 
ments  ranging  from  mild  to  severe  (i.e.  GLOC)  may 
be  possible.  These  techniques  combined  with  ad¬ 
vances  in  relating  brain  models  and  cognitive  func¬ 
tion  improve  the  probability  that  EEG  monitoring 
can  be  a  viable  technology  in  the  cockpit.  The 
largest  barrier  to  cockpit  EEG  has  and  will  remain 
the  development  of  sensors  for  incorporation  into  the 
helmet. 

Techniques  for  monitoring  the  physical  fatigue  lev¬ 
els  have  matured  and  now  include  metrics  extracted 
from  cardiovascular  parameters  [5,  8],  the  elec¬ 
tromyogram  (EMG)  [9],  and  the  breathing  patterns 
of  the  pilot. 


6  Integration  into  the  Electronic 
Crewmember 

This  paper  argues  that  systems  for  both  modeling 
and  monitoring  the  pilot  and  the  life  support  system 
should  be  integrated  into  the  electronic  crewmember 
(Figure  3)  in  essence  developing  what  Dr.  Mark  Dar- 
rah  of  Gentex  Corporation  described  as  the  Human 
Avionics  System,  Development  of  an  updated  model 
of  the  physiological  and  cognitive  state  incorporat¬ 
ing  data  fusion  techniques  and  other  AI  technologies 
such  as  neural  networks  and  fuzzy  logic  is  needed 
to  provide  a  composite  representation  of  the  human 
crew  component  to  the  electronic  component 

Combining  first  principle  and  empirical  models  with 
real-time  monitoring,  and  integrating  this  data  into 
a  state  model  of  the  pilot  can  optimize  both  the  life 
support  systems  of  the  aircraft  as  well  as  provide 
information  to  the  electronic  crew  component. 
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Figure  3.  Schematic  of  real-time  feedback  control  of 
an  adaptive  life  support  system  providing  Gz  protec¬ 
tion,  The  system  is  constantly  updating  and  refining 
the  state  model  of  the  pilot  and  providing  pilot  sta¬ 
tus  information  to  the  electronic  crew  member.  The 
initial  pilot  state  is  hosed  on  first-principle  and/or 
empirical  models.  For  subsequent  missions  a  new 
pilot  model  and  state  estimator  is  stored. 


7  Conclusions 

Any  system  involving  human-electronic  crew  inter¬ 
action  must  be  capable  of  providing  a  real-time 
model  of  the  physiological  and  cognitive  state  of  the 
pilot,  the  ability  to  undertake  trend  analysis  on  that 
state,  and  predict  the  state  for  the  future  course  of 
the  mission. 

The  aircrew  life  support  system  must  be  an  inte¬ 
gral  part  of  the  human-electronic  crew  system,  both 
to  maximize  human  crew  performance  and  provide 
crew  state  information  to  the  electronic  crew  com¬ 
ponent. 
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GROUP  DISCUSSIONS 


1.  INTRODUCTION 

Group  Discussions  were  convened  during  the  final  two  days  of  the  meeting  for  the  purpose  of 
developing  understanding  on  new  issues.  Participants  were  divided  into  multi-disciplinary 
groups,  each  with  a  designated  leader,  and  with  a  set  of  pre-selected  issues  to  consider  and  to 
evaluate.  This  section  summarizes  reports  of  the  group  leaders  to  final  Plenary  Session  of  the 
meeting.  Also  included  are  copies  of  cartoons  drawn  by  Mike  Busbridge  to  illustrate  progress 
and  status  in  the  area,  since  the  first  meeting  in  1988. 

2.  PROCEDURE 

The  programme  sought  to  make  the  workshop  productive  by  developing  a  shared  understanding 
on  new  issues.  Accordingly,  the  aim  of  the  Group  Discussions  was  to  enable  the  emergent  issues 
to  be  addressed  in  a  systematic,  and  yet  informal  manner,  with  the  widest  participation  from  all 
attendees.  In  order  to  facilitate  this  process,  all  participants  were  invited  to  begin  to  identify  and 
analyse  key  issues  from  the  start  of  the  meeting.  The  aim  was  to  develop  personal  understanding 
of  other  participants’  views  and  opinions,  through  both  formal  and  informal  discussion  (i.e.  in 
the  bar),  or  whenever  seemed  appropriate.  Ideally,  these  discussions  would  be  taken  forward 
throughout  the  meeting,  and  not  held  to  the  end,  when  the  organised  Group  Discussions  took 
place.  Whilst  interested  in  the  views  of  individuals,  the  organisers  sought  analyses  which 
synthesised  the  range  of  perspectives  and  positions  that  were  available  at  this  multi-national, 
multi-disciplinary  meeting. 

2.1  Identification  of  Issues 

Participants  were  particularly  encouraged  to  address  areas  of  uncertainty,  or  problems  requiring 
resolution.  The  organisers  were  interested  in  arguments  rather  than  statements  of  individual  s 
positions.  It  was  briefed  that  issues  should  be  approached  in  the  form  of  questions  using  the 
simple  imperatives  Why?  What?  Which?  How?  Who?  Where?  and  Whenl  Also,  it  was 
considered  helpful  to  frame  issues  in  terms  of  contrasting  positions,  such  as  “A  or  B?”  or  C 
versus  D?” 

2.2  Recording  of  Issues 

Participants  were  provided  with  two  forms.  These  sought  to  help  capture  analysis  of  issues.  The 
first  form  was  for  reporting  the  order  of  priority  of  the  important  issues  identified  by  the  group. 
The  second  form  was  for  recording  justification  information  for  high  priority  issues.  This 
required  information  on  the  following; 

□  Implications  of  the  issue  for  Human-Electronic  Crew  (H-EC)  teamwork; 

□  Factors  affecting  the  issue; 

□  Other  relevant  issues; 

□  Relevant  knowledge  i.e.  lessons  learnt,  current  practice,  methods  and  techniques; 

□  Potential  directions  i.e.  requirements,  alternatives,  choices,  priorities,  and  cost/benefits. 

The  forms  were  used  to  capture  the  results  of  the  Group  Discussions  and  to  record  individual 
analyses.  Acetates  of  the  forms  were  used  as  overheads  to  assist  Group  Leaders’  summary 
presentations  in  the  final  Plenary  Session.  The  reports  of  the  group  discussions  provided  in  this 
section  are  based  on  the  content  of  the  leaders’  briefings. 


201 


3.  INITIAL  ISSUES 

The  meeting  Call  Notice  identified  a  series  of  issues  relevant  to  Human-Electronic  Crew 
teamwork.  Participants  were  encouraged  to  consider  these  issues,  and  others  that  emerged 
during  the  paper  presentations,  in  particular  those  arising  from  the  Ke3niote  Address  .  It  was 
suggested  that  issues  that  had  arisen  at  previous  meetings  might  still  be  relevant.  During  the 
meeting,  a  list  of  emerging  issues  was  collated  and  displayed  for  consideration.  Participants 
were  encouraged  to  add  to  this  list  as  the  meeting  proceeded.  The  following  is  a  list  of  the  initial 
issues  presented  for  consideration  by  the  participants  at  the  1997  meeting,  including  issues  raised 
in  1994. 

3.1 1997  Theme  Issues:  The  Right  Stuff 

□  What  are  the  core  qualities  that  the  Electronic  Crewmember  must  possess? 

□  How  does  one  estimate  the  amount  of  software  code  involved? 

□  What  are  the  key  software  modules? 

□  What  is  necessary  to  ensure  the  modules  function  symbiotically? 

□  What  is  sufficient  functionality  within  the  Electronic  Crewmember  to  satisfy  the  human 
operator  requirements? 

3.2  Additional  1997  Issues 

The  following  issues,  statements  and  questions  were  raised  by  individual  participants  for 
consideration  in  advance  of  the  meeting: 

3.2.1  Issues 

□  Are  common  architectures  possible? 

□  Is  knowledge  re-use  possible  across  a  wide  area? 

□  Are  conventional  validation  methods  applicable  to  decision  support? 

3.2.2  Statements  on  Certification:  Agree  or  disagree? 

□  Achieved  design  quality  is  the  degree  of  adherence  to  design  requirements. 

□  Certification  activities  should  be  related  to  the  individual  stages  of  the  relevant  life  cycle. 

□  Certification  is  a  progressive  activity  conducted  throughout  both  the  software  and  system  life 
cycles 

□  Certification  is  the  process  of  checking  the  justification  of  design  choices  rather  than  of  the 
design  rationale  per  se. 

□  Design  is  the  traceable  maintenance  of  relevant  documentation:  such  documentation  is  the 
design. 

□  Formal  engineering  methods  show  a  greater  utility  to  Knowledge  Based  System  engineering 
than  the  use  of  like  methods  in  software  engineering. 

□  Provided  the  software  process  model  can  be  understood,  the  software  can  be  certified. 

□  Software  correctness  (degree  of  programmed  validity  of  rules,  knowledge  etc.)  is  equivalent 
to  safety  -  the  designed  degree  of  correctness  being  made  appropriate  to  known  requirements 
of  the  software’s  application  domain. 

□  Verification  is  concerned  with  checks  to  promote  good  design  practice;  Validation  is 
concerned  with  checks  concerned  with  interpretations  of  fitness  for  purpose. 
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3.2.3  Questions 

□  User-centred  or  use-centred  design? 

a  Adaptive  systems  or  adaptive  interfaces? 

□  Adaptive  or  adaptable . ? 

□  Function  allocation  or  integration? 

□  Physical  or  logical  specification? 

□  Specified  or  emergent  system  properties? 

□  Dependency  versus  autonomy? 

□  Feedback  or  feedforward  control? 

3.3  1994  Theme  Issues:  Can  We  Trust  the  Team? 

□  Do  current  development  activities  address  the  teaming  issues? 

□  Are  there  some  types  or  categories  of  decisions  or  actions  that  the  Human-Electronic  Team 
should  never  be  trusted  with? 

□  What  oversight  checks  should  be  placed  on  the  Team? 

□  How  does  the  Team  communicate  with  the  higher  authorities? 

□  Are  there  other  issues  besides  teaming  which  are  crucial  to  the  operational  application  of  the 
Electronic  Crewmember  concept? 

3.4  1994  Keynote  Issues; 

□  Who  should  be  the  team  leader  -  the  mission  computer  or  the  human? 

□  What  types  of  teams  should  the  system  emulate? 

□  How  do  we  ensure  that  the  team  samples  we  experiment  on  are  representative? 

□  What  human  characteristics  should  we  allow  for  in  our  team? 

□  How  many  humans  should  there  be  in  the  aircraft? 

□  How  much  should  the  team  members  trust  each  other? 

3.5  1994  High  Priority  Emergent  Issues: 

□  To  implement  an  EC,  shouldn't  we  adopt  a  stepped  approach  to  improve  existing  and  new 
cockpit  modalities? 

□  Can  we  justify  not  sharing  technology? 

□  Without  models  of  human  decision  making,  can  we  build  adaptive  intelligent  systems? 

□  What  authority  structures  and  EC  roles  should  there  be  in  a  multi-user  environment? 

□  How  do  we  measure  and  predict  success? 

□  How  do  we,  as  a  community,  demonstrate  the  benefits  of  EC? 

□  How  do  we  provide  guidelines  on  design  and  test  procedures  for  the  team  as  a  unit? 

□  How  do  we  provide  a  design  framework  to  include  the  role  of  the  operator? 

□  How  do  we  ensure  that  the  H-EC  team  functions  when  faced  with  the  unpredicted  mission? 

□  How  do  we  know  that  the  design  is  based  on  the  right  operational  tasks  and  context? 

□  What  are  the  barriers  to  our  problems  with  designing  an  effective  pilot/EC  interface,  and  how 
do  we  get  around  them? 

□  Where  are  we  going? 

□  What  are  the  stages? 

□  What  is  the  standard? 

□  How  much  interoperability? 

□  How  do  we  employ  aiding  in  design? 

□  How  do  we  achieve  effective  Pilot  Vehicle  Interface  implementation? 
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□  Should  we  start  slowly  and  incrementally? 

□  Why  has  there  been  no  development  in  EC  in  the  last  4  years? 

a  How  do  we  select  representative  samples  of  humans  for  use  in  H-E  crew  research  and  as  a 
model  on  which  to  build  systems? 

□  What  are  the  teams  and  their  rigorous  definitions? 

□  What  are  the  objective  design  and  development  criteria? 

□  How  would  an  EC  be  flight  and  combat  certified? 

□  How  does  one  ensure  that  information  of  the  highest  priority  is  delivered  to  the  pilot  soon 
enough  to  make  a  difference  to  mission  effectiveness  and  survivability? 

3.6  Earlier  Themes; 

□  1990:  Is  the  Team  Maturing? 

□  1988:  Can  they  Work  Together? 

4.  GROUP  DISCUSSIONS  REPORTS 

4.1  Group  1 

Membership: 

Tim  Barry 

Frank  Hoeppner 

Tom  Metzler 

Chris  Miller 

Plus  notes  from  Thierry  Joubert 

Priority  Issues: 

1.  How  do  we  provide  cost/  benefits  analysis  and  justification  for  ECs?  It  was  noted  that  the  2 
and  3'**  listed  1997  Initial  Issues  were  both  cost/benefit  or  money  issues. 

□  This  requires  consideration  of  ‘What  will  sell?’  and  of  ‘How  can  we  know?  On  what  to 
sell?’,  if  not  the  dream  (i.e.  now),  we  should  consider  how  can  we  sell  pieces  towards  the 
dream?  The  question  was  posed:  ‘Can  EC  sell  relative  to  more  traditional  approaches. 
It  was  noted  that  costs  are  accurate  measures,  whereas  benefits  differ  from  domain  to 
domain.  It  might  be  possible  to  sell  EC  on  its  own,  or  on  the  coat  tails  of  something  else 
e.g.  free  flight.  It  was  also  noted  that  integration  can  save  money,  but  only  to  the 
integrator  or  customer. 

2.  What  practical  steps  are  required  toward  achieving  real  world  ECs  (evolution)? 

□  It  was  noted  that  an  evolutionary  approach  implies  the  provision  of  simple,  limited 
functionality.  To  do  more  may  be  limiting  to  acceptance.  But  it  was  in  breadth  of 
integration  where  the  big  payoff  resided.  The  required  steps  (and  functionality)  would 
differ  across  domains  and  missions.  It  was  also  necessary  to  understand  the  path  toward 
the  goal.  For  future  direction,  it  would  be  necessary  to  consider  the  difference,  if  any, 
between  a  simple  decision  aid  and  a  baby  EC. 

3.  How  do  we  ensure  that  steps  are  compatible  with  the  EC  dream  for  system  integration,  and 
ultimately  system  certification? 

□  To  ensure  that  steps  are  on  the  path  to  the  goal,  we  need  to  understand  the  growth  path, 
and  we  may  need  to  give  up  the  EC  name.  Generic  EC  architectures  can  help  this 
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progression.  Human  Machine  Interface  and  functionality  may  need  to  be  considered  as 
separable  issues.  Integration  of  components  was  the  sine  qua  non  of  EC.  Again, 
integration  can  save  money,  but  only  to  the  integrator  or  customer.  It  was  relevant  to 
consider  shared  knowledge  bases,  and  shared  task  and  world  models.  Future  work  should 
include  the  sharing  of  requirement  specifications  for  EC.  There  was  a  need  to  ‘black 
box’  the  specification/requirement  for  disparate  people  to  work  forward.  But  the  Pilot’s 
Associate  did  this!  We  need  to  ask  why  it  was  not  successful? 

4.  How  do  we  determine  the  required  levels  of  authority?  There  was  considered  to  be  a  need  to 
overcome  the  ‘Pilot  in  control’  barrier.  The  required  analysis  needed  to  be  conducted  across 
application  domains  and  across  the  mission. 

With  regard  to  the  Initial  Issue,  concerning  the  required  core  capabilities  of  EC,  it  was  noted 
that  the  core  functionality  was  that  of  a  ‘back-seater’  providing  information  filtering  and. 
prioritisation. 

On  this  issue,  Thierry  Joubert  noted  that  in  the  Co-pilot  Electonique  rules,  the  first  was  to  help 
the  pilot  to  anticipate,  and  the  second  was  that  the  EC  must  know  and  respect  it’s  own  limits  (i.e. 
real  intelligence).  So,  the  EC  must  be  robust.  Good  models  were  needed  for  the  domains  that 
produce  problems  (e.g.  threats,  failures).  The  design  of  the  Man  Machine  Interface  should  be 
carefully  done  with  a  good  cognitive  approach.  He  believed  that  a  multi-agent  supervised 
architecture  could  be  standardised.  EC’s  way  of  thinking  should  ‘mimic’  some  particular  human 
pilot,  providing  a  natural  dialogue  that  avoided  surprises.  For  future  directions,  Thierry  Joubert 
recommended  work  on  “any  time  algorithms”  for  real  time  robustness,  and  on  “high  fidelity 
simulation”  for  combat  robustness. 

4.2  Group  2 

Membership: 

Simon  Goss 

John  Reising 

Gordon  Semple 

Joe  VonHolle 

Dave  Williamson 

Robert  Zanconato 

(aided  by  Asimov,  Lucas,  Orwell) 

Priority  Issues: 

1 .  Is  the  mission  the  proper  analytic  focus,  rather  than  task  analysis? 

□  An  implication  of  this  statement  was  considered  to  be  that  analysis  of  H-EC  Teamwork 
required  additional  categorisation  (boxes).  Psychological  analysis  was  considered  to  be  at 
the  task  level.  The  pilot’s  role  was  to  give  the  mission  a  ‘better  spin’.  Will  the  EC 
‘emerge’  in  the  super-sub-systems? 

□  Consideration  of  EC  emergence  was  believed  to  have  implications  for  the  integration  of 
intermediate  steps,  for  synergy  arising  from  a  ‘chunked’  development,  and  for  the  overall 
steer  of  distributed  development  of  an  H-EC  system.  The  possibility  of  implementation 
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‘windows’  for  emerging  functions  was  raised  as  an  influencing  factor  affecting  this  issue, 
along  with  the  requirements  of  ‘plug  and  play’  standards. 

2.  What  is  the  required  architecture  for  delivery? 

□  The  possibility  was  raised  that  EC  may  be  implemented  in  a  wearable  form,  and  as  a 
retro-fit.  Wearable  computing  was  under  development  for  foot-soldiers.  Commercial 
portable  devices  were  wearable.  Issues  arising  from  consideration  of  matching  the  brain 
to  boxes,  and  standards  of  inter-connectionism  were  identified  as  related  to  the  questions 
of  delivery  architecture. 

3.  What  dynamic  is  needed  to  provide  the  required  change  in  command? 

With  reference  to  the  workshop  lead  issue/question  about  EC  core  qualities,  it  was  noted  that 
George  Lucas  had  provided  the  following  suggestions:  ""Extend  my  sensors;  follow  my  lead;  do 
what  I  ask;  cover  my  butt  and  give  sensible  suggestions  which  match  the  urgency  of  the 
moment....  ”  It  was  suggested  that  we  should  not  allow  EC  to  be  viewed  as  “Big  Brother”,  nor  as 
a  psycho-analyst.  Rather,  EC  should  be  seen  as  a  “kick-butt,  ice  water  cool  warrior”. 

4.3  Group  3 

Membership: 

Lt  Andrew  (Gadget)  Davies 
Dr  Forster 
Peter  Gosling 
Dave  Harry 
Howard  Howells 
Reiner  Onken 

Priority  Issues; 

1.  How  do  we  resolve  the  competing  design  constraints  and  requirements  as  drivers  of  system 

development? 

□  This  issue  was  considered  to  have  implications  for  the  need  for  evolution  or  revolution  in 
the  H-EC  Team;  for  whether  to  automate  or  assist;  for  the  need  for  complete,  robust,  and 
flexible  systems;  and  for  the  implications  of  technology  driven  solutions.  It  was  thought 
that  the  issue  was  influenced  by  consideration  of  system  architectures,  proposed 
platforms,  technology  and  cost.  Standardisation  was  a  related  issue.  Consideration  was 
needed  of  knowledge  of  front  line/operational  requirements,  comparison  of  different 
systems,  and  lessons  learnt  from  the  Pilot’s  Associate  (PA)  and  Rotorcraft  Pilot’s 
Associate  (RPA)  programs.  Future  work  should  seek  to  maximise  benefit  from  the  cross¬ 
pollination  of  ideas  arising  from  PA/RPA,  Crew  Assistant  Military  Aircraft  (see  paper  8), 
Copilote  Electronique  (see  paper  7),  etc. 

□  How  do  we  determine  the  appropriate  task  allocation  i.e.  human/machine  split  or 
teamwork? 

□  This  issue  was  considered  to  have  H-EC  Team  implications  for  human-only 
responsibilities,  for  overlap  of  task  support,  for  any  unsupported  tasks,  and  for  dynamic 
changes  dependent  on  temporal  requirements.  It  was  thought  to  be  influenced  by  system 
tmst,  cost  and  training,  certification  requirements,  and  cognitive  workload  and  mission 
complexity.  The  requirements  of  crew  acceptance,  system  perceived  conflicts,  and 
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cooperation  /  teamwork  provided  related  issues.  Useful  knowledge  for  determining  the 
appropriate  task  structure  arose  from  understanding  about  implemented  and  designed 
systems  (e.g.  RPA/PA),  from  the  automation  versus  assistance  debate,  and  from  research 
on  co-operative  working. 

2.  What  approach  to  H-EC  system  control? 

□  This  was  considered  to  have  implications  for  the  level  of  control  required,  the  method  of 
control,  and  the  nature  of  the  control  loop.  Cost  and  complexity  were  thought  to  be  key 
factors  influencing  system  control  considerations.  Crew  training  was  a  related  issue. 
Lessons  learnt  from  PA/RPA  should  be  valuable.  The  way  forward  was  to  test  and 
demonstrate  control  options  to  determine  the  R&D  required. 

3.  Is  the  route  to  certification  through  standards? 

4.  What  are  the  alternative  methods  to  human  emulation? 

4.4  Group  4 

Membership: 

David  Barr 
George  Brander 
Mike  Busbridge 
HarmutH 
Gunter  Kroh 
Iain  MacLeod 
Tim  Normanton 

Priority  Issues: 

EC/Human  teaming  was  identified  as  the  key  issue. 

This  has  implications  for  the  7  Cs  of  teaming  in  the  H-EC  Team,  namely: 

□  Cohesiveness 

□  Coordination 

□  Communication 

□  Control 

□  Concurrency 

□  Collaboration 

□  Cybernation 

It  also  has  implications  for  the  3  Ts,  namely: 

□  Tactics, 

□  Training 

□  Technology 

Factors  influencing  EC/Human  teaming  were  differences  between  humans  and  machines, 
command  style,  and  that  only  a  human  can  be  ‘situationally  aware’.  Other  related  issues  included 
reliability,  quality,  training  and  certification.  Consideration  should  be  given  to  the  relevance  of 
knowledge  of  training  for  teams  and  selection.  Potential  future  directions  for  developing 
understanding  included: 
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□  How  does  the  pilot  configure  or  personalise  the  team? 

□  How  to  train  teams? 

□  The  need  for  a  new  type  of  pilot? 

In  an  individual  submission,  George  Brander  identified  EC/Human  Teamwork  as  the  key  issue. 
He  believed  that  the  main  implications  were  that  EC  must  be  commander  of  a  crew  of  EC 
functions  or  modules.  The  pilot  must  delegate  tasks,  responsibilities  and  ways  of  reporting. 
George  Brander  advised  that  we  must  have  a  system  configured  to  best  suit  the  mission  and  the 
pilot;  this  involved  training  issues  and  certification.  Influencing  factors  were  that  only  the  pilot 
could  have  situational  awareness  and  anticipation  (feed-forward?).  EC  functions  must  be 
reliable,  predictable,  but  also  adaptable  by  the  pilot  commander  to  serve  his/her  needs.  A  related 
issue  was  that  symbiosis  was  the  product  of  a  good  team.  He  considered  that  emergence  was  the 
problem  of  unwanted  factors  arising.  He  noted  that  mission  rehearsal  should  be  used  to  test 
tactics  and  EC  responsibilities.  For  potential  directions,  George  Brander  identified  architectural 
implications  as  important  i.e.  “How  does  the  pilot  configure  or  personalise  his  team?”  He  noted 
that  this  question  was  about  more  than  just  configuring  the  displays.  Research  was  needed  on 
how  to  train  teams  (i.e.  individual  skills  versus  team  skills).  On  certification,  he  advised  that  this 
should  be  of  the  platform  (e.g.  aircraft),  in  the  way  that  the  Navy  assesses  a  ship’s  capability. 

Mike  Busbridge  also  submitted  separate  notes  indicating  a  priority  flow  from  the  team,  to  the 
pilot-in-charge,  to  delegation,  with  trust  and  training  as  related  issues.  He  advised  that  the  EC 
must  have  the  ability  to  handle  its  delegated  tasks  autonomously,  in  a  timely  manner.  Also,  EC 
must  have  knowledge  of  its  limitations  to  pass  over  the  problem  when  it  gets  beyond  its 
competence.  Mike  Busbridge  suggested  that  it  would  be  useful  to  know  what  task(s)  it  (EC)  is 
competent  in  now,  perhaps  as  a  list.  He  noted  that  the  most  useful  tasks  are  the  most  difficult  to 
implement.  For  future  work,  Mike  Busbridge  described  a  team  trainer,  with  a  focus  for  the  EC  on 
the  7  Cs  (Group  4  issues),  and  a  focus  for  the  pilot  on  individual  characteristics. 

4.5  Group  5 

Membership: 

Floyd  Glenn 
Karmen  Guevara 
Rod  King 
Ron  Small 

Priority  Issues: 

1 .  How  to  provide  a  multi-disciplinary  approach? 

□  The  need  for  a  multi-disciplinary  approach  in  developing  the  H-EC  Team  was  thought  to 
require  a  continual  re-visiting  of  requirements,  using  a  holistic  perspective.  Key  factors 
influencing  the  provision  of  a  multi-disciplinary  approach  were  considered  to  be 
organisational  and  academic  segregation,  and  unpredictability  in  the  use  of  the  system. 
Another  related  issue  was  the  approach  taken  to  prototyping. 

2.  How  to  separate  function  and  use? 
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□  This  issue  was  believed  to  have  implications  for  unused  and  unusable  functionality  in  the 
H-EC  team.  A  key  factor  was  the  separation  of  the  user  from  the  developer,  and  from 
Human  Computer  Interface  specialists. 

3.  When  to  provide  advisory  versus  supervisory  support? 

4.  How  to  combine  evidence? 

4.6  Group  6 

Membership: 

Ulf  Berggrund 
Gert  Dorfel 
Dieter  Scheithauer 
HVaic 

Priority  Issues: 

1 .  For  functional  requirements  analysis: 

□  How  to  classify  mission  tasks  and  pilot  behaviour? 

□  What  constitutes  flexibility  for  the  EC? 

□  What  safety  classification? 

2.  For  the  route  to  certification: 

□  What  traceable  and  adaptable  methods  and  tools  from  other  technologies  can  be  used  in 
certification? 

□  What  new  methods  can  be  used  for  compliance  demonstration  for  the  remaining  tasks? 

□  What  is  the  feasibility  and  cost  effectiveness  of  these  new  technologies? 

□  How  to  start  to  invent  these  methods  and  tools? 

3.  For  reusability: 

□  How  to  provide  a  general  classification  of  appropriate  means  to  accomplish  a  certain  task 
category? 

□  How  to  get  a  common  understanding  of  a  modular  system  architecture. 

The  primary  issue  -  functional  requirements  analysis  —  was  considered  to  have  implications  for 
the  effectiveness  of  H-EC  Teamwork,  particularly  with  regard  to  the  following: 

□  mission  accomplishment, 

□  safety,  and 

□  user  acceptance  (feel  good  factor). 

The  key  factors  influencing  functional  requirements  analysis  were  considered  to  be  as  follows: 

□  the  mission  tasks, 

□  follow-on  or  new  (from  scratch)  system  development,  and 

□  the  required,  allowable  reaction  time. 

Other  relevant  issues  in  functional  requirements  analysis  arose  from  consideration  of  the 
requirements  for  system  analysis,  cognitive  task  analysis,  and  organisation  analysis. 

Knowledge  that  was  considered  to  be  relevant  for  functional  requirements  analysis  included: 

□  understanding  and  classification  of  pilot  behaviour 

□  understanding  of  flexibility,  and 

□  classification  of  mission  tasks. 

Potential  directions  identified  for  future  work  included  the  classification  of  tasks  and  pilot 
behaviour  especially  with  regard  to  the  requirements  of  testing  and  evaluation. 
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1.  At  Christmas  1985,  Sam  arrives  first  to 
collect  his  gift,  proudly  bearing  an 
aeroplane,  with  a  helicopter  held  discretely 
behind  his  back.  James  comes  empty 
handed,  but  with  a  boat  also  held  out  of 
sight.  Henri  arrives  on  his  bike.  Hans 
arrives  hopefully.  The  presents  are  all 
puzzles.  But  there  are  no  pictures  to  guide 
them. 


2.  In  1988,  Sam  demonstrates  progress 
with  his  big  puzzle.  He  has  put  it  together 
without  the  aid  of  a  picture.  With  the  help 
of  some  simple  rules,  the  comers  and  sides 
are  done.  But  the  rest  are  only  roughly 
sorted.  He  thinks  he  may  need  some  more 
‘know-how’  help  to  put  them  in  place. 
Hans  is  in  festive  mood,  enthusiastically 
assisted  by  James.  They  have  both  made 
good  progress  with  their  puzzles.  Henri  is 
seen  to  be  dashing  about  on  his  bike. 


3.  By  1997,  Sam  has  decided  that  he 
wants  his  picture  to  be  a  land  battle.  He  has 
found  some  parts  that  seem  to  work,  but 
they  don’t  fit  together.  James  proudly 
presents  his  completed  puzzle.  It  is,  not 
surprisingly,  a  naval  scene.  Hans  lets  us 
see  his  simple  solution;  it’s  a  peaceful, 
civilian  scene.  Henri  is  working  on  two 
concurrent  solutions  to  his  puzzle.  One  is  a 
bomber.  The  other  is  a  fighter  aircraft. 


4.  Looking  forward,  with  2000 
foresight,  nothing  is  certain.  But,  Hans, 
under  a  new  pennant,  confidently  expects  to 
be  flying  his  simple,  achievable  solution.  He 
rightly  deserves  a  feather  in  his  cap,  and  the 
title  Super-Hans.  Henri,  also  mit  plume, 
offers  the  prospect  of  a  new,  big  puzzle  to 
solve.  But  he  has  not  yet  revealed  his 
picture. 
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